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Abstract 


The  Army  Research  Laboratory  (ARL)  was  activated  at  the  beginning  of 
fiscal  year  1993.  Since  then,  ARL  has  operated  in  an  environment  of  continu¬ 
ing  stress,  caused  by  a  variety  of  factors.  Like  many  other  "corporate" 
research  laboratories,  it  has  been  subject  to  economic  constraints  and  press¬ 
ures,  leading  to  downsizing,  consolidation,  infrastructure  reduction,  and 
outsourcing.  As  a  military  laboratory,  it  has  also  been  significantly  affected 
by  the  end  of  the  Cold  War. 

To  cope  with  this  changing  environment,  ARL  has  undertaken  a  variety  of 
management  initiatives:  in  the  current  jargon,  it  has  been  reinventing  itself. 
This  report  summarizes  the  current  status  of  these  ongoing  initiatives.  It 
begins  with  a  discussion  of  the  fundamental  reinvention  of  the  research 
effort  at  ARL — the  Federated  Laboratory — and  then  discusses  several  others 
that  directly  or  indirectly  support  it.  The  report  concludes  with  a  summary 
appraisal,  in  "report  card"  form,  of  the  reinvention  efforts  over  the  past  five 
years. 
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Foreword 


This  report  is,  in  some  sense,  the  culmination  of  almost  three  decades  of 
working  towards  the  improvement  in  the  management  processes  and  tech¬ 
niques  of  Army/DOD  research  organizations  in  general,  and  of  the  Army 
Research  Laboratory  (ARL)  and  its  predecessor,  the  Harry  Diamond  Labo¬ 
ratories  (HDL)  specifically.  As  such,  the  views  and  opinions  expressed 
herein  are  strictly  those  of  the  author  and  do  not  in  any  way  represent  an 
official  position  of  ARL,  the  Army,  or  the  Department  of  Defense. 
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1.  Introduction  and  Background 


The  Army  Research  Laboratory  (ARL)  has  undertaken  a  variety  of  man¬ 
agement  initiatives,*  several  beginning  even  before  its  activation  almost 
five  years  ago.  Some  of  these  initiatives  began  under  the  auspices  of  pro¬ 
grams  such  as  the  National  Partnership  for  Reinventing  Government  (for¬ 
merly  the  National  Performance  Review— NPR),  some  were  in  response  to 
the  Government  Performance  and  Results  Act  of  1993  (GPRA),  and  some 
were  inspired  by  the  familiar  need  for  cost-effectiveness  in  an  era  of  tight 
budgets.  Some  of  these  initiatives  have  broken  new  ground  in  several  areas 
of  federal  R&D  management,  such  as  partnering  with  the  private  sector, 
business  planning,  and  performance  evaluation. 

This  report  is  intended  to  assess  what  we  have  accomplished,  where  we 
have  come  from,  and  where  we  need  to  go  next.  This  assessment  is  timely, 
since  Congress  is  now  turning  its  attention  to  this  question.  Congress 
wants  to  know  what  benefits  (and  in  particular,  what  savings)  have  been 
derived  from  the  NPR  waivers  program,  whether  Total  Quality  Manage¬ 
ment  (TQM)  has  had  any  noticeable  impact,  whether  GPRA  is  being  imple¬ 
mented  successfully,  and  so  forth. 


REKVRTRG 

Govmmi 


Parts  of  ARL's  efforts  have  received  outside  recognition:  During  1998,  ARL 
received  three  of  Vice  President  A1  Gore's  Hammer  Awards  from  the  NPR. 
Two  of  those  awards  were  for  initiatives  discussed  in  this  report:  the  Feder¬ 
ated  Laboratory  and  the  Alternative  Personnel  Demonstration  System. 
(The  third  Hammer  was  given  in  response  to  a  customer's  nomination  for 
some  exceptional  service  performed  in  a  technical  program.) 


Even  without  the  obvious  encouragement  of  Congressional  interest  and 
other  outside  scrutiny,  it  is  appropriate  to  pause  after  more  than  five  years 
of  effort  for  an  assessment.  This  report  looks  at  all  our  initiatives,  puts 
them  in  perspective,  provides  a  "warts  and  all"  evaluation  of  our  activities, 
and  indicates  where  we  need  to  go  next. 


The  report  begins  with  an  evaluation  of  the  Federated  Laboratory 
(FedLab)  initiative,  the  fundamental  reinvention  that  ARL  has  undertaken. 
The  rest  of  the  report  describes  the  other  initiatives,  which  are  either  in  di¬ 
rect  support  of  FedLab  or  in  general  are  efforts  to  improve  the  overall  envi¬ 
ronment  in  which  FedLab  functions.  These  other  initiatives  include  the 
"Open  Laboratory,"  ARL's  work  as  a  National  Reinvention  Laboratory  un¬ 
der  the  NPR,  Business  Process  Reengineering  (BPR),  and  our  work  in  the 
Department  of  Defense  (DoD)  Laboratory  Quality  Improvement  Program 
(LQIP),  which  includes  our  Alternative  Personnel  System  Demonstration. 
Other  efforts  that  have  a  more  indirect  relation  to  FedLab  include  our 
GPRA  initiatives,  benchmarking,  and  an  in-depth  investigation  into  the 
causes  and  control  of  overhead  costs.  These  various  reinvention  initiatives 
can  be  viewed  in  terms  of  supporting  the  four  principles  of  the  NPR  (table 
1).  The  report  concludes  with  a  view  of  the  application  of  TQM  to  a 
research  organization. 


*Descriptians  of  all  the  management  initiatives  discussed  herein  are  linked  to  the  ARL  homepage  at  http://zv3.arl.mil/ 
mgtinit/. 
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Table  1.  NPR 
principles. 


1.1 


ARL 

initiatives 

Cutting 

red 

tape 

Putting 

customers 

first 

Empowering 

employees 

Cutting 
back  to 
basics 

Federated  Laboratory  (FedLab) 

— 

— 

X 

_ 

Open  Laboratory 

X 

— 

X 

_ 

National  Performance  Review 
Reinvention  Lab  (waivers, 
including  LQIP  waivers) 

X 

X 

— 

Business  Process  Reengineering 
(BPR) 

X 

— 

— 

X 

Alternative  Personnel  System 
Demonstration  (Laboratory 
Quality  Improvement  Program, 
LQIP) 

X 

X 

Government  Performance  and 
Results  Act  (GPRA)  (business 
planning  and  performance 
evaulation) 

X 

Benchmarking 

— 

— 

— 

X 

National  Academy  of  Science/ 
Research  Roundtable  Overhead 
study 

— 

— 

X 

Quality  programs  — 

(customer  value,  HEARTS,*  PQA+) 

X 

— 

— 

*An  experiential  learning  program  (HEARTS  =  Honesty ,  Ethics ,  Accountability, 
Respect ,  Trust,  and  Support). 

Presidential  Quality  Award. 


ARL's  History 

ARL's  genesis  is  found  in  seven  formerly  independent  Army  laboratories, 
one  of  which  traced  its  ancestry  as  far  back  as  1816.  These  were  as  follows: 

Atmospheric  Sciences  Laboratory,  White  Sands  Missile  Range,  NM 

Ballistics  Research  Laboratory,  Aberdeen  Proving  Ground,  MD 

Electronic  Technology  and  Devices  Laboratory,  Ft.  Monmouth,  NJ 

Harry  Diamond  Laboratories,  Adelphi,  MD 

Human  Engineering  Laboratory,  Aberdeen  Proving  Ground,  MD 

Materials  Research  Laboratory,  Watertown,  MA 

Vulnerability  Assessment  Laboratory,  White  Sands  Missile  Range,  NM 

In  addition,  to  these  organizations,  the  Army  Research  Office  (ARO)  will 
become  a  part  of  ARL  at  the  beginning  of  FY99.  ARO,  located  in  the  Re¬ 
search  Triangle  area  of  North  Carolina,  is  the  Army's  principal  link  to  the 
academic  basic  research  community.  Through  grants  to  universities,  it  sup¬ 
ports  a  wide  variety  of  fundamental  investigations  in  areas  of  interest  to 
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the  Army.  It  is  being  combined  with  ARL  to  bring  a  greater  synergy  and 
mutual  leveraging  of  the  Army's  in-house  and  out-of-house  research 
programs. 

When  ARL  was  established,  the  fundamental  research  parts  of  a  number  of 
other  Army  organizations  were  also  folded  into  the  new  organization. 
Though  the  genealogy  is  complex  (see  appendix),  the  current  form  of  ARL 
is  the  result  of  a  combination  of  events  occurring  in  the  1987-1993  time 
frame.  These  included  the  Defense  Management  Review,  conducted  by  the 
Office  of  the  Secretary  of  Defense  (OSD)  in  1989-1990,  which  led  to  the 
Army's  Lab  21  study  in  1990,  which  recommended  the  establishment  of  an 
Army  corporate  research  lab.  Following  these  two  studies  there  were 
several  others,  among  them  the  Federal  Commission  on  the  Consolidation 
and  Conversion  of  Defense  Laboratories,  the  Base  Realignment  and 
Closure  Commission  (specifically,  BRAC  88  and  BRAC  91),  and  a  study  by 
the  Board  on  Army  Science  and  Technology  (BAST)  of  the  National 
Research  Council.  Based  on  the  results  of  all  these  studies,  the  final  form 
for  ARL  was  determined,  and  in  October  of  1992  ARL  was  activated,  with 
two  main  campuses  at  Adelphi  and  Aberdeen,  an  outdoor  test  facility  at 
White  Sands,  an  Atlanta  site  on  the  Georgia  Tech  campus,  and  two  smaller 
detachments  colocated  with  NASA  laboratories  at  their  Lewis  and  Langley 
Research  Centers.  The  site  at  Watertown  was  closed,  with  its  activities 
being  relocated  to  the  Aberdeen  campus,  and  the  presence  at 
Ft.  Monmouth  moved  to  Adelphi. 

ARL's  charter  dictated  that  it  concentrate  solely  on  basic  and  applied 
research  (6.1  and  6.2  programs)  and  weapons  analysis  (6.6),  giving  up  to 
other  organizations  all  the  development,  engineering,  and  low-rate  initial 
production  work  that  had  been  done  in  several  of  its  predecessor  organiza¬ 
tions.  Its  primary  customers  became  the  Research,  Development  and  Engi¬ 
neering  Centers  (RDECs)  of  the  commodity  commands  of  the  Army  Mate¬ 
riel  Command  (AMC). 

The  objective  of  these  various  efforts  was  to  improve  the  way  fundamental 
research  is  performed  in  an  Army  environment.  In  an  era  of  downsizing 
and  budget  cuts  occurring  simultaneously  with  a  changing  world  situation 
and  demands  for  an  increasing  future  reliance  on  technology,  it  became 
obvious  to  the  senior  leadership  that  the  Army's  research  enterprise 
needed  to  be  streamlined  and  focused. 

This  arrangement  is  analogous  to  the  industrial  model  of  a  central  labora¬ 
tory  supporting  the  R&D  centers  in  a  large  corporation's  product  divisions 
(fig.  1).  Thus,  ARL  became  the  Army's  "corporate  laboratory." 


1.2  Environment 


The  environment  in  which  ARL  operates  has  two  salient  features: 
extreme  pressure  on  resources  and 
heavy  mission  demands. 
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Industry 


Figure  1.  ARL's  role  is 
that  of  a  corporate 
laboratory  in  industry: 
both  to  gather  and  to 
generate  technology. 


ARL  is  subject  to  the  same  kinds  of  pressures  felt  by  its  private  sector  cous¬ 
ins  in  the  R&D  community.  When  resources  become  scarce,  long-term 
research  becomes  a  target  for  cutting,  since  (by  definition)  its  impact  is  not 
felt  for  many  years.  In  general,  R&D  is  difficult  to  defend:  its  products  are 
difficult  to  measure,  its  outcomes  cannot  be  quantified  in  advance,  and 
results  are  often  more  serendipitous  than  predictable.  Further,  the  two-  or 
three-decade  lag  between  investment  and  payoff  means  that  leadership 
possessing  extraordinary  foresight  is  needed  to  defend  against  budget  cuts 
when  funding  is  tight. 

As  a  research  organization,  ARL  is  susceptible  to  these  kinds  of  pressure; 
as  a  military  research  organization,  ARL  is  further  susceptible  to  the 
changes  that  resulted  from  the  end  of  the  Cold  War.  Since  the  fall  of  the 
Berlin  Wall,  ARL  (and  its  predecessor  organizations)  has  been  reduced  45 
percent  in  personnel  and  50  percent  in  funding  (fig.  2),  and  the  end  of  this 
drawdown  is  not  yet  in  sight.  Various  external  forces  continue  to  press  for 
even  more  closures,  consolidations,  and  outsourcing  of  R&D,  including 
pressures  to  increase  cross-service  integration  of  programs,  to  seek  "dual 
use  solutions  to  materiel  problems,  and  to  use  commercial  specifications, 
standards,  and  products. 

In  the  midst  of  these  challenges,  the  Army's  senior  leadership  gave  ARL  an 
entirely  new  mission.  In  response  to  the  Desert  Storm  experience.  General 
Gordon  R.  Sullivan,  the  previous  Chief  of  Staff  of  the  Army,  directed  that 
AMC  proceed  to  develop  the  technology  to  "digitize  the  battlefield."  In 
other  words,  AMC  must  devise  systems  that  will  enable  real-time  situ¬ 
ational  awareness  for  battlefield  commanders  at  all  levels.  Such  awareness 
demands  wireless,  near-instantaneous  communication  vertically  and  hori¬ 
zontally,  with  total  fusion  of  all  relevant  information  (intelligence,  weather 
and  terrain  data,  logistics  information,  etc),  which  must  then  be  displayed 
in  a  readily  comprehensible  format.  ARL  was  designated  to  develop  the 
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Figure  2.  Decline  in 
ARL's  personnel  and 
financial  resources 
(before  ARL  activation, 
data  reflect  ARL's 
baseline  determined 
from  predecessor 
organizations). 
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scientific  underpinning  and  hand  off  the  necessary  technology  to  the  Com¬ 
munications  and  Electronics  Command  (CECOM)  to  design,  build,  and 
field  the  systems.  ARL's  current  situation  is  thus  a  classic  case  of  having  to 
"do  more  with  less." 


1.3  Reinvention  and  R&D 

These  times  of  dramatic  change  require  dramatic  new  ways  of  doing  busi¬ 
ness.  Both  in  the  corporate  world  and  in  government,  "reinvention"  and 
"reengineering"  are  hot  topics;  books  on  these  subjects  have  made  it  to  the 
tops  of  various  bestseller  lists.  Reinvention  is  all  around:  in  the  private  sec¬ 
tor,  innumerable  companies  have  "reinvented"  themselves  (sometimes 
with  dramatic  results),  and  in  the  federal  government,  the  Vice  President's 
National  Partnership  for  Reinventing  Government  (NPR)  initiative  is 
aimed  at  reinvention.  Increasing  effectiveness  and  efficiency  has  become 
critical  for  survival. 

The  problem  with  such  initiatives  is  that  they  risk  becoming  faddish; 
everyone  jumps  on  the  bandwagon  whether  it  makes  sense  or  not. 
Although  the  NPR  office  has  documented  many  success  stories,  many 
claimed  reinventions  are  only  working  at  the  margins.  For  example,  rein¬ 
venting  the  way  your  mail  is  delivered  may  be  nice,  but  it  does  not  contrib¬ 
ute  to  the  fundamental  mission  of  your  organization  (unless  you  are  the 
Postal  Service).  The  joke  about  rearranging  the  deck  chairs  on  the  Titanic 
comes  to  mind  (especially  if  an  organization  is  in  real  danger  of  sinking). 

In  R&D  it  is  especially  difficult  to  find  opportunities  for  substantive  rein¬ 
vention;  R&D  is  simply  too  complex.  While  there  is  ample  opportunity  to 
reinvent  the  various  support  processes  in  an  R&D  organization,  reinvent¬ 
ing  how  the  laboratory  performs  its  fundamental  mission  is  quite  another 
story.  This  is  the  task  ARL  has  set  itself  in  devising  the  FedLab. 
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2.  The  Federated  Laboratory:  The  Reinvention  of  R&D 

The  Federated  Laboratory,  or  FedLab,  is  the  fundamental  reinvention  of 
how  ARL  performs  its  mission.  This  concept  of  operations  was  instituted 
as  a  means  to  deal  with  the  problem  of  conducting  a  new  mission  assign¬ 
ment  at  the  same  time  that  our  resources  were  being  reduced.  The  new 
mission — to  develop  the  scientific  underpinnings  to  digitize  the  future 
battlefield  presented  multiple  challenges:  an  important  consideration 
was  the  existing  exceptionally  strong  private  sector  competence  (directed 
at  civilian  applications),  which  ARL  could  not,  and  indeed  should  not,  try 
to  compete  with.  Thus,  ARL  developed  the  concept  of  an  intimate 
partnering  with  the  leaders  of  the  private  sector  efforts.  In  addition  to  al¬ 
lowing  us  to  fulfill  our  mission  responsibilities  with  a  dramatically 
downsized  staff,  this  approach  has  the  advantage  that  it  strongly  leverages 
the  commercial  sector's  large  investment  in  expertise  and  facilities  for  the 
Army's  benefit. 


2.1  Background 

When  General  Sullivan  asked  ARL  to  provide  CECOM  with  the  basic  tech¬ 
nology  required  to  develop  the  future  digitized  battlefield,  the  ARL  Direc¬ 
tor,  Dr.  John  Lyons,  inventoried  his  capabilities  in  this  area.  He  found  that 
although  several  researchers  scattered  throughout  ARL  were  working  in 
various  technical  areas  relevant  to  the  problem,  the  number  was  far  from 
the  critical  mass  of  expertise  required  for  this  difficult  technical  problem. 
Moreover,  it  was  evident  that  the  private  sector  was  far  ahead  of  the  mili¬ 
tary  in  developing  and  fielding  wireless  digital  communications — as  evi¬ 
denced  by  the  number  of  people  walking  around  the  streets  with  cellular 
phones  held  to  their  ears.  Moreover,  with  new  wireless  devices  appearing 
in  the  stores  almost  monthly,  it  was  equally  clear  that  the  field  was  moving 
so  fast  that  there  was  no  possibility  of  starting  up  an  effort  from  scratch 
and  becoming  a  leader  in  the  field.  There  was  not  enough  money,  people, 
or  time  to  do  that. 

However,  it  also  should  be  noted  that  there  are  fundamental  differences 
between  commercial  and  military  technologies.  For  example,  in  the  spe¬ 
cific  case  of  digitized  communications,  the  commercial  sector  has  the  ad¬ 
vantage  of  using  fixed  infrastructures  in  relatively  benign  environments. 
The  military,  on  the  other  hand,  must  provide  similar  wireless  communica¬ 
tions  plus  all  the  other  intelligence  and  data  fusion  tasks,  in  an  environ¬ 
ment  of  electronic  jamming  and  hostile  fire,  for  a  customer  that  is  the 
equivalent  of  a  medium  size  city  on  the  move.  Thus,  the  approach  to  the 
digitized  battlefield  problem  would  have  to  bring  the  commercial  and 
military  perspectives  together. 

In  light  of  all  those  considerations,  ARL  decided  to  move  the  existing  tech¬ 
nical  elements  within  ARL  into  several  focused  groups  and  then  go  where 
the  expertise  already  existed — the  private  sector. 

Traditionally,  this  would  have  been  accomplished  through  some  sort  of 
contractual  process.  This  approach,  however,  has  several  major  disadvan¬ 
tages:  First,  with  contracts,  which  are  "arms  length"  procurement  vehicles. 
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ARL  would  not  be  able  to  specify  the  research  objectives  in  advance  and 
then  modify  them  as  the  results  emerge.  Second,  we  would  be  unable  to  be 
active  participants  in  the  technical  work.  Rather,  we  would  be  able  only  to 
write  scopes  of  work  and  issue  contracts  based  on  them,  which  then  (if  we 
were  lucky)  would  yield  technology  that  might  be  relevant  to  the  ex¬ 
tremely  complex  and  sophisticated  problems  that  the  Army  would  face  on 
tomorrow's  battlefield.  Such  a  process  would  not  enable  ARL  to  act  as  a 
"smart  buyer"  for  the  Army,  since  only  a  limited  amount  of  in-depth  tech¬ 
nical  knowledge  would  be  available  in  house.  What  was  needed  was  a 
new  way  to  do  our  business — a  reinvention. 

The  answer  appeared  in  the  form  of  a  new  authority  granted  to  the  DoD 
under  10USC2358:  Cooperative  Agreements  (CAs,  not  to  be  confused  with 
Cooperative  R&D  Agreements,  or  CRADAs,  which  are  in  no  way  related). 
The  most  important  aspect  of  CAs  is  that  they  allow  an  intimate  and  sub¬ 
stantial  relationship  between  the  government  and  the  vendor.  This  meant 
that  ARL  would  be  able  to  form  true  partnerships  with  the  private  sector: 
to  jointly  plan  and  execute  technical  programs,  jointly  evaluate,  assess,  and 
report  on  the  work  accomplished,  and  redirect  the  work  as  necessary. 

The  potential  offered  by  this  new  authority  was  immediately  evident.  ARL 
would  be  able  to  join  with  the  private  sector  to  produce  the  required  tech¬ 
nology  and,  in  the  process,  expand  the  in-house  team  to  allow  the  Army  to 
efficiently  integrate  these  new  technologies  into  its  acquisition  system. 
Thus  was  born  the  Federated  Laboratory  concept  of  operation,  FedLab. 

2.2  What  is  FedLab? 

FedLab  can  be  thought  of  as  a  collection  of  geographically  distributed, 
"virtual"  laboratory  divisions,  augmenting  the  capabilities  of  the  ARL  di¬ 
rectorates.  These  divisions  work  in  the  technical  areas  relevant  to  building 
a  scientific  foundation  for  the  digital  battlefield.  FedLab  involves  integrat¬ 
ing,  under  ARL  leadership  and  direction,  programs  in  the  private  sector 
with  those  already  existing  within  ARL.  Funding  under  the  CA  authority 
is  provided  by  the  government;  this  is  not  a  cost-sharing  arrangement. 
CAs  are  no-fee /no-profit  arrangements. 

The  original  design  of  the  FedLab  concept  was  envisioned  as  follows: 

•  Several  different  technology  areas  all  related  to  the  overall  thrust  of  digitiz¬ 
ing  the  battlefield  were  defined. 

•  For  each  area,  a  consortium  would  be  formed  to  execute  the  program  in 
partnership  with  ARL. 

•  A  consortium  would  consist  of  at  least  one  industry  partner  as  the  consor¬ 
tium  lead,  one  major  research  university  partner,  and  one  Fiistorically 
Black  College  or  University  or  Minority  Institution  (HBCU/MI),  which 
would  receive  at  least  10  percent  of  the  consortium's  funding. 

•  Each  consortium's  activities  would  be  directed  by  a  Consortium  Manage¬ 
ment  Committee  (CMC)  formed  by  senior  representatives  of  all  the  part¬ 
ners  and  chaired  by  a  senior  ARL  technical  manager  as  the  Cooperative 
Agreement  Manager  (CAM). 
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•  The  duration  of  the  CAs  would  be  sufficiently  long  that  the  fruits  of  the 
basic  and  applied  research  could  be  realized. 

•  So  as  to  enhance  technology  transfer  between  ARL  and  the  private  sector 
partners  to  the  greatest  possible  extent,  a  requirement  of  successful  bid¬ 
ding  consortia  would  be  to  engage  in  long-term  technical  staff  rotations, 
with  ARL  personnel  working  in  our  partners'  labs  and  our  partners'  per¬ 
sonnel  working  in  our  lab. 

•  For  enhanced  communication  among  each  consortium's  members,  each 
member  would  have  to  be  capable  of  video-teleconferencing. 

This  approach  was  intended  not  only  to  bring  together  the  best  of  the  pri¬ 
vate  and  the  public  sectors  in  responding  to  a  technically  demanding 
challenge,  but  also  to  respond  to  several  of  the  other  external  pressures 
placed  on  ARL:  it  would  increase  outsourcing,  help  ARL  to  seek  dual-use 
solutions,  and  use  commercial  standards  and  products. 

FedLab  was  also  to  be  a  research  multiplier."  The  science  and  engineering 
(S&E)  staff  would  be  enhanced  via  the  working  relationships  with  the  best 
of  the  private  sector  (including  the  staff  exchanges).  FedLab  would  take 
advantage  of  existing  state-of-the-art  facilities,  and  encourage  the  con¬ 
struction  of  new  industry  facilities.  We  also  expected  that  IR&D  (Indepen¬ 
dent  Research  and  Development)  would  be  refocused  to  support  FedLab 
projects.  Commercial  technologies  would  be  adopted  and  adapted  to  the 
military  environment.  And  of  greatest  importance,  we  would  be  able  to 
build  an  in-depth  knowledge  base  and  competence  in  the  technology, 
despite  the  overall  reduction  envisioned  for  the  DoD  in-house  work  force. 
We  would  be  able  to  do  more  with  less! 

It  is  also  satisfying  to  note  that  in  the  Senate  Armed  Services  Committee 
report  on  the  FY95  Defense  Authorization  Act,  the  FedLab  concept  was 
highly  praised,  and  additional  funds  were  recommended  to  support  it.* 


2.3  Implementation  of  the  FedLab  Concept 

After  the  concept  was  developed  and  approvals  gained  from  the  chain  of 
command,  the  actual  implementation  of  FedLab  began  in  December  of 
1994  with  the  issuance  (in  hard  copy  and  on  the  World  Wide  Web)  of  a 
Broad  Agency  Announcement  (BAA)  that  introduced  and  described  the 
concept,  and  specified  the  requirements  outlined  above.  Five  technical 
areas  were  defined  for  which  consortia  could  compete.  These  were 

•  advanced  distributed  simulation, 

•  telecommunications /information  distribution, 

•  software  and  intelligent  systems. 

The  Army  s  initiative  to  create  an  open,  federated  laboratory  system  is  an  innovative  and  forivard- 
thinking  approach.  The  committee  supports  the  competitive  selection  of  laboratories  from  industry  and  academia  to 
work  with  the  Army  Research  Laboratory  to  meet  the  Army  research  needs  across  a  wide  range  of  technologies  The 
committee  recommends  an  increase  of  $20.0  million  in  PE  6U02A  to  accelerate  this  effort." 

Source:  National  Defense  Authorization  Act  for  FY 1995,  SASC  Report,  June  14, 1994. 
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•  advanced  and  interactive  displays,  and 

•  advanced  sensors. 

It  was  planned  to  offer  five-year  CAs  to  one  consortium  in  each  area,  with 
a  government  option  to  extend  the  relationship  noncompetitively  for  an 
additional  three  years.  The  planned  five-year  investment  was  estimated  to 
be  approximately  $175M. 

After  a  draft  of  the  BAA  was  announced  in  November  1994,  two  day-long 
presolicitation  conferences  were  held,  one  on  each  coast,  to  explain  the 
details  and  answer  questions  that  potential  bidders  might  have  and  to  take 
comments  for  the  final  BAA  to  be  released.  Approximately  400  people 
attended  each  of  these.  To  introduce  potential  partners  to  the  ARL  staff  and 
programs,  we  held  an  open  house  in  January  1995  at  our  Adelphi  site, 
where  the  technical  work  would  be  done.  Another  400  people  came 
to  walk  through  our  labs,  see  actual  ongoing  work,  and  talk  to  our 
researchers.  There  were  also  many  thousands  of  "hits"  on  our  web  page 
announcement. 

We  were  very  pleased  that  these  efforts  yielded  37  proposal  packages,  and 
that  the  consortia  that  were  assembled  to  bid  consisted  of  the  leaders  of 
digital  technologies  in  the  private  sector.  Interestingly,  rather  than  the 
minimum  three-member  consortia  required,  most  of  the  bidding  groups 
contained  from  a  half  dozen  to  a  dozen  and  a  half  members. 

During  the  spring  of  1995,  a  meticulous  proposal  evaluation  process  took 
place  under  the  leadership  of  the  Army  Research  Office  (ARO)  in  North 
Carolina.  Over  80  people  were  involved  in  the  month-long  evaluation. 

However,  before  the  five  awards  could  be  announced.  Congress  reconsid¬ 
ered  its  support  of  FedLab.  Our  concept  had  become  caught  up  in  (and, 
some  believe,  confused  with)  issues  revolving  around  Federally  Funded 
Research  and  Development  Centers  (FFRDCs).  Despite  last-minute  efforts 
to  separate  our  concept  from  the  FFRDC  issue.  Congress  reduced  the  fund¬ 
ing  and  directed  that  only  three  technical  areas  be  supported,  and  that  the 
three-year  extension  option  be  dropped. 

2.4  Current  Status 

The  winning  consortia  were  announced  on  16  January  1996.  The  surviving 
three  consortia  with  their  membership  and  funding  are  as  follows: 

Advanced  and  Interactive  Displays 

Members:  Rockwell  International  (industry  lead) 

Sytronics,  Inc. 

North  Carolina  A&T  Univ.  (HBCU /MI) 

Univ.  of  Illinois 

Microelectronics  Center  of  North  Carolina 

Funding:  $4.260M  in  FY97,  $4.364M  in  FY98;  approximately  $25M  over  five  years 

(HBCU/MI  =  9.3  percent) 

CAM:  Bernard  M.  Corona,  410-278-5916 
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Advanced  Sensors 


Members:  Lockheed  Sanders  (industry  lead)  ERIM 

Clark  Atlanta  Univ.  (HBCU /MI)  GTRI 

Univ.  of  New  Mexico  (HBCU /MI)  MIT 

Lockheed  Missiles  and  Space  Co.  Univ.  of  Miaryland 

Texas  Instruments  Univ.  of  Michigan 

Ohio  State  Research  Foundation  Stanford  Univ. 

Funding:  $6.912M  in  FY97,  $8.828M  in  FY98;  approximately  $50M  over  five  years 

(HBCU/MI  =  10.8  percent) 

CAM:  John  Miller,  301-394-5000 

Telecommunications/Information  Distribution 


Members:  Lockheed  Sanders  (industry  lead)  Bellcore 

Howard  Univ.  (HBCU/MI)  GTE  Labs 

Morgan  State  Univ.  (HBCU/MI)  MIT 

City  College  of  New  York  (HBCU/MI)  Univ.  of  Delaware 

Univ.  of  Maryland  Motorola 


Funding:  $6.533M  in  FY97,  $8.610M  in  FY98;  approximately  $47M  over  five  years 

(HBCU/MI  =  12.0  percent) 

CAM:  Dr.  John  W.  Gowens  II,  301-394-1722 

This  effort  currently  involves  87  scientists  and  engineers  (S&Es)  from  in¬ 
dustry,  71  from  ARL,  and  92  faculty,  120  graduate  students,  and  11  post¬ 
doctoral  fellows  from  academia.  Four  different  ARL  directorates  are 
participating. 

The  benefits  of  FedLab  go  in  both  directions.  The  Army  gains  the  private 
sector  s  manufacturing  expertise  and  related  IR&D  and  academic  research 
programs.  Our  FedLab  partners  gain  a  better  understanding  of  Army- 
unique  research  and  of  the  soldier's  materiel  requirements.  In  addition,  to 
date  the  private  sector  has  invested  $5.3M  in  new  facilities  plus  an  addi¬ 
tional  $5.9M  in  internal  coinvestment  for  use  in  FedLab  programs. 

The  CAs  have  been  in  force  for  two  and  half  years.  In  that  time,  the  consor¬ 
tia  organized  themselves,  formulated  their  program  plans  for  FY96,  and 
began  work.  Each  program  was  divided  into  several  technical  thrusts  or 
factors,  each  of  which  contains  a  number  of  tasks.  Each  factor  is  led  by  an 
external  partner,  and  each  task  has  an  external  principal  investigator  and 
several  associate  partners,  including  an  ARL  investigator.  Figure  3  shows 
an  example  of  this  structure  for  the  Telecommunications  and  Information 
Distribution  consortium. 

Each  consortium  had  its  first  annual  two-day  technical  symposium  during 
January  1997,  at  which  the  results  of  its  first  year's  work  were  displayed. 
Attendance  varied  among  the  three  symposia  from  180  to  350  people. 
There  were  oral  presentations,  poster  sessions,  and  live  demonstrations,  all 
of  which  showed  a  remarkable  technical  effort  with  some  very  fine  results. 
In  addition,  144  technical  papers  were  published  and  14  technology  transi¬ 
tion  contracts  established  with  the  RDECs  and  the  Training  and  Doctrine 
Command  (TRADOC). 
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Figure  3.  Sample  of  research  plan  structure,  showing  program  divided  into  technical  factors  (left 
column),  each  containing  several  tasks  (listed);  other  columns  show  roles  of  consortium  members 
(Telecommunications  and  Information  Distribution  consortium). 

The  FY97  program  was  completed,  and  the  results  were  presented  in  Feb¬ 
ruary  1998.  This  year  one  single,  week-long  symposium  was  held  with  all 
three  consortia  reporting  out  at  the  same  time.  The  FY98  program  is  now 
under  way 

2.5  Assessment  and  Future  Directions 

Not  unexpectedly,  in  the  initiation  of  FedLab,  some  administrative  and 
cultural  problems  arose.  The  administrative  problems  were  fairly  minor, 
mainly  concerned  with  meshing  three  very  different  operating  systems 
(government,  industry,  and  academia)  in  terms  of  how  funds  were 
handled  and  how  personnel  were  managed.  The  cultural  problems  re¬ 
volved  around  the  very  different  approaches  that  industry  and  academia 
take  towards  accomplishing  tasks,  the  first  being  strongly  driven  by  dead¬ 
lines  and  deliverables,  and  the  second  more  by  investigation  and 
publishability.  However,  this  "culture  shock"  has,  for  the  most  part,  worn 
off  and,  all  things  considered,  is  not  a  major  problem. 

Program  contents  have  also  required  some  readjustment;  tasks  and  goals 
were  redefined  as  the  consortia  began  to  get  a  better  appreciation  of  the 
competencies  of  their  members.  It  also  became  apparent  that  the  ARL  in- 
house  contributions  were  very  different  in  each  of  the  three  areas.  For  ex¬ 
ample,  in  the  sensors  area,  ARL  has  a  very  strong  program  and  thus  takes 
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a  leading  role  in  that  consortium;  in  telecommunications,  however,  our  in- 
house  program  is  very  small,  and  so  we  have  become  more  of  a  follower 
than  a  leader  in  this  area  from  the  standpoint  of  technical  output.  How¬ 
ever,  it  has  not  been  a  problem  for  the  three  consortia  to  operate  somewhat 
differently;  rather,  it  is  a  sign  of  the  flexibility  of  the  FedLab  structure.  In  all 
cases,  though,  leadership  in  the  management  of  the  consortia  resides  at 
ARL. 

One  observation  made  after  the  first  year's  three  technical  symposia  was 
that,  while  a  lot  of  excellent  work  has  been  accomplished,  the  integration 
of  the  various  projects  into  a  unified  technical  program  has  not  yet  taken 
place.  Although  this  is  understandable  for  the  first  year,  attention  has  been 
focused  on  this  goal  during  the  second  year.  Such  integration  is  critical  to 
accomplishing  the  ultimate  goals  of  FedLab. 

It  is  intended  to  display  the  progress  towards  this  integration  at  the  third 
symposium,  scheduled  for  February  1999.  Again,  there  will  be  a  single 
symposium  with  all  three  consortia  participating.  However,  this  time  we 
will  feature  a  number  of  cross-consortia  joint  papers,  as  well  as  a  demon¬ 
stration  in  the  form  of  a  wargame  vignette  that  will  merge  the  technology 
outputs  of  the  three  consortia  into  a  single  scenario  of  real-time  situational 
awareness  and  data  fusion  and  display. 

Another  area  that  will  require  more  emphasis  is  the  number  of  staff  rota¬ 
tions.  The  BAA  specified  as  a  goal  that  20  percent  of  the  S&Es  working  in 
each  area  should  be  on  long-term  (one  to  one  and  a  half  years)  rotation.  At 
present,  this  all-important  feature  of  the  program  has  not  reached  the  tar¬ 
geted  percentage.  However,  currently  there  are  34  rotational  assignments 
into  ARL  from  our  partners  and  17  assignments  of  ARL  researchers  to  our 
partner's  labs. 

An  additional  challenge,  sometimes,  is  dealing  with  employees  of  our 
partners  on  rotation  to  ARL  who  are  foreign  nationals.  Controlling  their 
access  to  parts  of  the  ARL  facilities  where  classified  work  is  going  on  has 
been  something  of  a  problem. 

Overall,  FedLab  has  assembled  a  lot  of  energetic  people,  working  with  en¬ 
thusiasm  and  doing  good  technical  work.  The  consensus  among  the  man¬ 
agers  involved  with  FedLab  is  that  on  a  scale  of  1  to  5,  after  two  and  half 
years,  it  gets  a  4+. 

If  we  believe  that  FedLab  is  on  the  right  track  and  will  be  successful  in 
helping  ARL  fulfill  its  mission,  the  obvious  question  is:  "What  next?"  Sev¬ 
eral  possibilities  are  currently  being  considered  relating  to  the  future  of 
FedLab: 

•  Can  we  recapture  the  two  technical  areas  that  were  not  supported  by  Con¬ 
gress  in  the  original  digitization  effort?  (Advanced  Distributed  Simulation 
and  Software  and  Intelligent  Systems  were  deleted  from  the  program.) 

•  Can  we  create  a  new  consortium  within  the  digitization  effort,  or  expand 
an  existing  one,  that  will  bring  a  private  sector  partner  to  work  in  the  new 
world-class  facilities  about  to  be  completed  at  our  Adelphi  campus? 
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•  Are  there  whole  new  technology  mission  areas  in  which  ARL  could  con¬ 
tribute  by  establishing  other  FedLab  systems  of  partnerships  similar  to 
what  has  been  done  for  the  problem  of  digitizing  the  battlefield? 

The  last  question  in  particular  is  very  tantalizing,  since  there  are  a  number 
of  critical  problems  facing  the  future  Army  to  which  a  massive  technologi¬ 
cal  assault  could  provide  the  solutions.  These  are  now  under  active  consid¬ 
eration,  but  are  not  yet  ready  for  a  public  discussion. 


reinventing 

GOVERNMENT 


We  are  exceptionally  proud  to  note  that  on  1  July  1998  the  Principal 
Deputy  Director  of  the  NPR  and  the  Assistant  Secretary  of  the  Army  pre¬ 
sented  ARL  with  Vice  President  A1  Gore's  Hammer  Award  for  the  FedLab 
reinvention.  Of  all  the  Hammers  that  have  been  given  by  the  Vice  Presi¬ 
dent  to  teams  of  federal  workers,  this  one  may  be  unique,  in  that  it  was  for 
the  reinvention  of  the  core  business  process  of  R&D,  rather  than  for  a  func¬ 
tional  support  process. 


As  noted  earlier,  the  FedLab  concept  is  the  fundamental  reinvention  that 
ARL  has  undertaken  to  enable  it  to  better  perform  its  mission  in  this  time 
of  decreasing  resources  and  increasing  requirements.  The  initiatives  de¬ 
scribed  in  the  following  sections  of  this  report  all  relate  to  or  support,  ei¬ 
ther  directly  or  indirectly,  this  FedLab  reinvention  of  the  R&D  process. 
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3.  The  Open  Laboratory 

The  Open  Laboratory  initiative  is  a  collection  of  operational  procedures 
designed  to  support  the  large  number  of  guest  researchers  that  ARL  is  hop- 
ing  to  attract  to  our  facilities.  It  also  includes  procedures  necessary  to  help 
our  people  going  on  long-term  rotations  to  other  research  organizations, 
both  to  our  FedLab  partners,  and  to  other  institutions  within  the  world¬ 
wide  scientific  community.  The  Open  Lab  is  designed  to  deal  with  the 
problems  that  naturally  arise  from  such  a  flow.  The  concept  of  the  guest  re¬ 
searcher,  both  into  and  out  of  ARL,  is  also  part  of  our  larger  vision  of  ARL 
as  an  intellectual  crossroads  and  meeting  place.  This  vision  derives  from 
the  belief  that  a  world-class  research  organization  must  be  open  to  both 
ideas  and  people  from  elsewhere  in  the  scientific  community. 

3.1  Description  of  the  Open  Lab  Initiative 

In  a  fall  1996  report  entitled  Endless  Frontier,  Limited  Resources:  U.S.  R&D 
Policy  for  Competitiveness,  the  Council  on  Competitiveness  (a  nonpartisan, 
nonprofit  forum  of  business,  university,  and  labor  executives)  discussed 
what  it  considered  a  new  approach  to  technological  innovation.  This 
approach  is  based  on  collaboration  among  industry,  academia,  and  the  fed¬ 
eral  laboratory  system,  which  are  the  principal  players  in  the  nation's 
research  and  development  enterprise.  According  to  the  report,  collabora¬ 
tion  often  includes  personnel  exchanges  to  maximize  the  exploitation  of 
resources  represented  by  the  skills  and  knowledges  of  the  employees  and 
the  unique  facilities  of  the  laboratories.  This  report  confirmed  the  need  that 
we  recognized  to  establish  an  "open  laboratory." 

The  Open  Laboratory  concept  was  identified  and  initiated  several  years 
ago.  ARL's  Director,  Dr.  John  Lyons,  came  to  ARL  in  1992  from  a  long 
career  as  Director  of  the  highly  acclaimed  National  Institute  of  Standards 
and  Technology  (NIST),  where  the  use  of  personnel  exchanges  was  so  well 
established  that  the  number  of  guest  researchers  was  almost  equal  to  the 
number  of  S&Es  on  the  staff.  Dr.  Lyons  touted  the  many  advantages  of 
having  a  flow  of  guest  researchers  coming  into  ARL  to  conduct  research 
using  ARL's  unique  equipment  and  facilities,  as  well  as  the  benefits  to  ARL 
and  its  employees  of  ARL  personnel  spending  time  at  other  research  facili¬ 
ties  in  industry,  academia,  or  government  agencies,  for  time  spans  ranging 
from  two  weeks  to  more  than  a  year. 

Becoming  an  open  laboratory  allows  ARL  to  realize,  to  a  large  extent,  its 
vision  of  itself  as  "an  intellectual  crossroads  for  the  technical  community." 
By  open  laboratory,  we  mean  a  research  facility  that  permits  easy  access  to  its 
facilities  by  researchers  from  outside  the  laboratory,  that  supports  its 
researchers  as  they  work  in  other  laboratories,  and  that  actively  encour¬ 
ages  the  exchange  of  researchers,  ideas,  information,  and  results  with  other 
laboratories. 

In  response  to  the  Director's  encouragement,  the  exchange  of  guest  re¬ 
searchers  in  and  out  of  ARL  increased  significantly.  In  1993, 137  guest  re- 
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searchers  came  to  ARL,  providing  25  full-time  equivalent  (FTE)  work- 
years;  only  28  ARL  S&Es  were  guest  researchers  at  other  organizations, 
amounting  to  5  FTEs  of  labor.  By  1996,  the  number  of  guest  researchers 
coming  into  ARL  had  almost  doubled  to  245  (66  FTEs),  and  the  number  of 
ARL  employees  going  to  other  organizations  as  guest  researchers  had  al¬ 
most  quadrupled  to  110  (51  FTEs).  Annual  goals  were  set  of  400  guest  re¬ 
searchers  coming  into  ARL  by  2001  and  300  researchers  going  out  of  ARL 
by  2002.  FedLab  was  an  additional  impetus  to  the  Open  Laboratory  con¬ 
cept,  because  it  specifically  made  provisions  for  personnel  exchanges  be¬ 
tween  members  of  our  partners'  staffs  and  ARL. 

However,  such  openness  brings  with  it  a  host  of  operational  problems. 
Some  of  these  are  related  to  security,  since  some  classified  work  goes  on  at 
ARL.  Other  problems  are  the  ordinary  issues  that  any  organization  would 
confront  when  trying  to  allow  strangers  to  come  and  go  with  minimal  re¬ 
strictions,  and  at  the  same  time  providing  support  for  their  technical  en¬ 
deavors  while  on  site.  Rotation  of  our  personnel  out  also  raises  as  many  is¬ 
sues  as  rotations  of  guest  researchers  in.  The  Open  Laboratory  initiative  is 
an  attempt  to  identify  these  various  issues  and  potential  problem  areas, 
and  solve  them  so  that  the  vision  of  an  intellectual  crossroads  can  be 
achieved. 

The  principal  areas  of  concern  in  bringing  the  Open  Lab  concept  to  fruition 
are  security,  use  of  the  libraries,  and  the  logistics  of  staff  rotations,  both  in 
and  out. 

3.2  Problems  and  Recommended  Solutions 
3.2.1  Security 

Emblematic  of  the  Open  Lab  is  the  absence  of  guards  at  ARL's  front  gate 
during  normal  working  hours.  In  the  Cold  War  era,  the  guard  post  was 
always  manned,  and  all  staff  and  visitors  were  checked  as  they  entered  the 
compound.  The  rear  gate  was  always  closed  except  during  rush  hours, 
when  a  guard  was  also  stationed  there.  Dr.  Lyons  felt  that,  much  like  NIST 
(which  also  has  an  unmanned  guard  booth  at  its  gate),  ARL  must  present 
an  initial  appearance  of  openness  if  true  openness  is  to  be  attained. 

However,  there  is  a  tension  between  the  benefits  of  openness  (maximum 
accessibility  is  essential  to  doing  good  science)  and  the  requirement  to  pro¬ 
vide  physical  security  and  protect  classified  work  and  materials.  Despite 
the  desirability  of  increasing  the  number  of  guest  researchers  at  ARL,  secu¬ 
rity  concerns  preclude  allowing  unlimited  access  to  all  who  walk  in  the 
gate.  The  clearance  process  is  intended  to  reconcile  the  opposing  goals  of 
security  and  openness.  However,  it  does  not  solve  all  problems.  For 
example,  the  law  specifically  limits  the  access  that  foreign  nationals  may 
have  to  defense  materials  and  computer  networks;  thus,  some  clearances 
simply  cannot  be  granted.  More  commonly,  clearances  are  not  received  in 
order  or  in  a  timely  manner,  usually  because  our  own  researchers  do  not 
understand  security  requirements  and  fail  to  provide  advance  notice  of 
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visitors  to  the  security  office.  This  results  in  the  need  to  escort  uncleared 
guest  researchers,  the  requirement  of  which  falls  to  the  host  researchers, 
distracting  them  from  their  main  duties. 

A  problem  in  the  past  has  been  the  lack  of  an  up-to-date,  accurate  database 
that  can  identify  all  guest  researchers  on  site.  Management  could  not  be 
assured  that  they  knew,  at  any  instant,  who  was  here.  This  caused  obvious 
security  problems,  not  to  mention  problems  of  legal  liability.  This  situation 
arose  both  from  insufficient  in-  and  out-processing  procedures,  and  from 
the  underutilization  by  the  host  researchers  of  those  processes  that  do  ex¬ 
ist.  In  general,  it  appears  that  the  security  procedures  that  do  exist  were 
being  enforced  inconsistently,  and  there  was  confusion  among  our  own 
S&Es  as  to  the  proper  procedures.  Much  improvement  has  been  made  in 
both  these  processes  and  their  implementation. 

Further  relief  to  this  problem  will  be  realized  with  the  installation  of  a 
comprehensive  database,  now  under  construction,  that  will  contain  all  the 
pertinent  information  about  every  employee,  contractor,  and  guest  re¬ 
searcher  at  any  of  ARL's  sites. 

As  was  found  with  most  of  the  problems  that  Open  Lab  presented,  the 
greatest  hurdle  to  overcome  was  a  culture  change  on  the  part  of  our  own 
staff.  Taking  the  guard  off  the  gate  has  been  compensated  for  by  the  use  of 
that  manpower  to  provide  a  higher  degree  of  security  inside  ARL  build¬ 
ings,  as  well  as  a  roving  external  guard  presence  in  vehicles.  Restricting 
uncleared  visitors  to  certain  routes  within  the  buildings  has  been  debated. 
A  team  of  foreign  nationals  working  at  the  Adelphi  site  has  already  been 
physically  segregated  in  a  laboratory  facility  specially  built  in  an  unclassi¬ 
fied  building  that  is  separate  from  the  main  buildings.  However,  the  ulti¬ 
mate  answers  to  this  problem  will  be  increased  emphasis  on  educating  our 
own  host  S&Es  on  their  responsibilities,  and  getting  them  to  take  these  re¬ 
sponsibilities  more  seriously. 

ARL  has  also  approached  this  problem  as  a  technical  challenge.  Both  on 
the  battlefield  and  in  the  laboratory,  computer  security  and  information  as¬ 
surance  have  become  a  special  problem.  ARL  has  become  a  lead  agency  in 
information  assurance  and  defensive  information  warfare  research,  and 
we  are  using  our  own  laboratory  computer  systems  as  a  test  bed  for  our  re¬ 
search  for  the  field  army.  Thus,  many  of  the  hardware  and  software  proto¬ 
cols  being  studied  are  being  applied  to  the  problems  presented  by  guest  re¬ 
searchers  at  ARL  facilities.  ° 

3.2.2  Libraries 

A  well-run,  well-stocked,  and  easily  accessible  technical  library  is  abso¬ 
lutely  essential  to  a  world-class  research  organization.  Within  the  con¬ 
straints  of  budget  and  manpower,  the  libraries  at  our  two  main  campuses 
have  continually  striven  to  provide  the  services  and  atmosphere  required. 
However,  FedLab  and  the  increasing  emphasis  on  guest  researchers  under 
Open  Lab  have  created  a  new,  if  not  unexpected,  set  of  problems.  The 
library  at  Adelphi  and  three  of  the  four  library  sites  at  Aberdeen  are  inside 
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secured  areas,  thus  causing  the  most  obvious  problem:  access  by  visiting 
researchers  whose  clearance  is  not  available.  However,  this  is,  by  far,  not 
the  only  problem.  What  should  be  the  policy  on  letting  nonemployees 
remove  materials  from  the  library?  What  level  of  service  (e.g.,  in  terms  of 
complex  and  expensive  literature  searches)  should  be  provided  to 
nonemployees?  This  question  is  especially  difficult  when  the  requester  is  a 
contractor's  employee  doing  a  search  for  work  not  relevant  to  ARL.  How 
do  interagency  library  exchange  agreements  relate  to  non- ARL  employ¬ 
ees?  Indeed,  how  do  the  library  staff  even  recognize  a  guest  researcher  if 
his  badge  is  not  readily  in  sight,  and,  assuming  he  can  be  identified,  how 
do  the  staff  members  know  what  projects  he  is  working  on  and,  therefore, 
authorized  to  use  the  library  assets  for? 

Since  libraries,  like  all  other  support  functions,  cost  money  and  require 
personnel  to  operate,  they  must  husband  their  resources:  throwing  the 
library's  doors  open  to  all  comers  is  not  a  practical  option.  Little  has  yet 
been  accomplished  to  deal  with  these  problems.  It  has  been  suggested  that 
library  cards  be  issued  to  identify  employees  and  different  categories  of 
guests.  Some  thought  has  been  given  to  physically  separating  classified 
and  restricted  materials  in  the  libraries  and  opening  the  unrestricted  por¬ 
tion  as  an  unclassified  area.  However,  again  the  most  basic  thing  that  must 
be  done  is  to  obtain  a  clear  understanding  of  the  situation  as  it  currently 
exists.  To  do  this,  a  library  committee  was  formed,  held  several  meetings, 
conducted  some  surveys  and  investigations,  and  issued  a  report.  We  are 
now  awaiting  a  decision  on  that  report  before  moving  on  to  implementa¬ 
tion  of  its  recommendations. 


3.2.3  Staff  Rotations 

The  problem  of  staff  rotations  is  especially  complex.  Indeed,  both  in  rotat¬ 
ing  a  guest  researcher  in,  or  one  of  our  own  staff  members  out,  we  usually 
affect  a  person's  entire  way  of  living:  spouse  and  spouse's  employment, 
children  and  their  schooling  requirements,  housing,  transportation,  etc. 
One  cannot  just  tell  someone  to  show  up  next  Monday  at  a  new  job  loca¬ 
tion  hundreds  or  thousands  of  miles  away  and  leave  it  at  that.  Although,  at 
least  in  theory,  the  private  sector  can  apply  more  pressure  on  its  workers  to 
take  long-term  assignments  away  from  home,  we  in  the  government  must 
persuade,  cajole,  and  offer  incentives  for  an  employee  to  disrupt  his  or  her 
life  so  drastically.  Some  significant  issues  for  people  considering  rotations 
to  laboratories  outside  ARL  are  the  regulations  on  reimbursement  for  long¬ 
term  travel,  family  impact  (such  as  working  spouses,  school-age  children, 
availability  of  quality  child  care,  health  care  plans,  etc),  and  the  effect  on 
promotion  consideration. 

The  approach  to  rotations  both  in  and  out  has  so  far  been  ad  hoc;  as  rota¬ 
tions  become  more  common,  more  formal  procedures  will  be  needed.  For 
the  most  part,  the  situations  have  been  handled  case  by  case,  without  well- 
thought-out  and  published  guidance.  Prior  planning  has  in  many  cases 
been  minimal.  The  travel  offices  at  the  two  main  ARL  campuses  handle 
their  processes  differently.  Travel  reimbursements  for  long-term  travel  as- 


17 


signments  have  not  been  timely,  and  with  the  regionalization  of  all  our 
finance  and  accounting  functions,  our  concern  in  this  area  has  increased. 
While  a  number  of  band-aid"  fixes  have  been  implemented  for  many  of 
these  problems,  a  unified  and  coherent  policy  needs  to  be  established  and 
published.  Publications  such  as  a  sponsor's  guide  for  hosts  of  visiting  re¬ 
searchers  and  a  welcome  package  for  the  visitors  are  desperately  needed, 
as  is  a  guide  for  those  rotating  out  on  long-term  assignments  in  our  part¬ 
ners'  labs. 

Finally,  there  can  be  significant  problems  on  reentry  for  an  ARL  staff  mem¬ 
ber  on  a  long-term  rotation.  Sometimes  the  researcher's  office  is  waiting  as 
it  was  left,  and  the  program  is  in  a  pause  mode.  In  this  case,  the  worker  just 
steps  back  into  the  picture  as  if  he  or  she  had  never  left.  On  the  other  hand, 
often  programs  are  completed  and  new  ones  started,  or  the  branch  may 
have  been  reorganized  or  physically  moved  to  new  quarters.  In  these 
cases,  utilization  plans  for  the  returning  worker  are  necessary  to  reinte¬ 
grate  the  person  back  into  the  ARL  community.  Unfortunately,  with  the 
various  stresses  on  managers  due  to  the  current  environment,  these  types 
of  plans  are  not  often  done  in  any  more  than  a  cursory  manner. 

3.3  Current  Status 

An  advisory  group  on  implementation  of  the  Open  Lab  was  established, 
including  both  technical  and  support  services  employees.  This  group 
reported  to  the  ARL  Deputy  Director,  who  was  responsible  for  assuring  the 
implementation  of  the  Open  Lab.  This  advisory  group  established  a  set  of 
definitions  of  terms  used  in  implementing  the  Open  Lab.  Because  of  the 
broad  range  of  functions  affected  by  the  Open  Lab,  the  advisory  group  rec¬ 
ommended  setting  up  subgroups  as  follows: 

•  a  resources  working  group  to  determine  the  costs  associated  with  rotations 
into  and  out  of  ARL; 

•  a  rotations  working  group  (1)  to  identify  the  issues  and  processes  needed 
to  make  rotations  attractive  for  ARL's  S&Es  and  (2)  to  develop  a  rotations 
manual; 

•  a  sponsors  working  group  to  develop  a  manual  for  ARL  sponsors  of  guest 
researchers;  and 

•  a  legal  working  group  to  identify  the  legal  ramifications  and  requirements 
involved  and  to  document  them  in  an  Open  Lab  Legal  Handbook. 

The  advisory  group  wrote  an  Open  Lab  concept  paper  identifying  some  of 
the  main  issues  to  be  considered  by  the  subgroups.  These  included  the 
need  to  develop  (1)  a  "welcome"  packet  for  guest  researchers  to  smooth 
their  arrival;  (2)  a  one-stop  guest  researcher  orientation  covering  all  in¬ 
processing  and  out-processing  procedures;  (3)  training  for  ARL  employees 
going  on  rotations  and  training  for  sponsors  of  guest  researchers;  (4)  plans 
for  managing  space/ computer /library/ clerical  needs;  (5)  means  for  main¬ 
taining  security  without  impeding  the  guest's  ability  to  work  effectively; 
and  (6)  ARL  policy  providing  rotation  incentives. 
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Implementation  of  the  recommendations  has  been  uneven,  mainly  because 
of  the  downsizing  of  the  support  offices  and  the  resulting  increased 
demands  on  their  time.  The  Open  Lab  Legal  Handbook  was  published  in 
August  1996,  and  the  Open  Lab  Rotation  Manual  was  published  in  July 
1996,  but  the  Sponsors  Guide  was  not  finalized,  and  the  resources  working 
group  dissolved  without  accomplishing  its  mission.  The  issue  of  security 
and  mobility  of  guest  researchers  is  still  being  addressed  and  is  being 
handled  on  an  individual  basis.  The  one-stop  guest  researcher  orientation 
and  the  training  for  sponsors  and  employees  considering  rotations  have 
not  been  developed,  so  the  administrative  burden  for  the  technical 
employees  is  considerable  and  may  be  a  barrier  to  meeting  the  Open  Lab 
goals.  This  is  also  a  direct  threat  to  achieving  the  staff  rotation  goals  of 
FedLab. 

While  there  are  many  benefits  to  the  Open  Lab  concept,  it  requires  a  cul¬ 
ture  change  that  may  be  difficult  for  some  people  to  accept.  The  implemen¬ 
tation  of  the  Open  Lab  concept  must  have  high-level  ownership  and  be  an 
integrated  effort,  with  policies  applicable  across  the  laboratory.  Orienta¬ 
tions,  training,  assistance  in  finding  challenging  assignments  for  guests 
and  for  ARL  employees  going  on  rotations,  and  easing  the  administrative 
burdens  of  the  Open  Lab  are  issues  that  have  been  identified  but  not 
resolved.  However,  the  rate  at  which  personnel  exchanges  have  increased 
in  the  last  few  years  is  encouraging.  This  suggests  that  as  the  issues  identi¬ 
fied  are  resolved,  the  rate  of  exchanges  will  increase  further,  and  ARL  can 
become  the  model  of  an  "open  laboratory." 
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4.  A  Reinvention  Laboratory  Under  the  National 
Partnership  for  Reinventing  Government:  Waivers 

With  the  downsizing  of  ARL's  staff,  the  impact  of  our  relocations  due  to 
BRAC,  and  the  inception  of  the  FedLab  initiative,  it  became  evident  that 
our  remaining  support  staff  would  have  more  and  different  responsibili¬ 
ties  per  person;  in  order  to  cope  with  the  new  situation,  they  would  need 
as  much  help  and  as  little  red  tape  as  we  could  arrange.  Being  an  NPR  re- 
invention  laboratory  enabled  us  to  obtain  waivers  to  a  wide  variety  of 
regulations  and  policies  that  were  either  irrelevant  or  duplicative,  or  in 
some  sense  overly  constraining  and  time  consuming.  While  no  one  waiver 
saved  a  large  amount  of  money,  we  have  been  able  to  save  a  great  deal  of 
"process  time,"  thereby  allowing  the  support  staff  to  serve  the  technical 
staff  more  efficiently,  flexibly,  and  responsively. 

4.1  Background 

In  February  1994,  ARL  was  designated  an  NPR  Reinvention  Laboratory. 
We  were  actually  grandfathered  into  the  program,  since  we  had  been,  in 
our  former  incarnation  as  Laboratory  Command,  a  participant  in  the  DoD 
Laboratory  Demonstration  Program.  (The  Lab  Demo  program  had  been 
established  in  1989  in  response  to  a  recommendation  of  the  Defense  Sci¬ 
ence  Board  1987  Summer  study.)  In  the  early  days  of  the  NPR  program, 
quite  frankly,  we  didn't  understand  it  very  well  and  didn't  think  much 
about  it,  except  that  using  the  Vice  President's  NPR  logo  seemed  to  add  a 
nice  touch  to  our  briefing  charts.  It  is  probably  also  fair  to  say  that  in  those 
early  days,  the  NPR  staff  did  not  quite  have  a  grip  on  their  program,  be¬ 
cause  we  got  precious  little  advice  from  them  in  terms  of  what  it  really 
meant  to  be  a  reinvention  laboratory.  Although  the  word  "waiver"  kept 
popping  up  (which  certainly  sounded  like  there  could  be  benefits  to  being 
a  reinvention  lab),  there  was  no  process  in  place  to  gain  waivers  from  rules 
and  regulations. 

Gradually  it  became  apparent,  however,  that  reinvention  had  more  to  do 
with  changing  the  way  an  organization  performed  its  mission  than  with 
how  many  waivers  one  could  obtain.  In  fact,  after  being  in  the  program  for 
three  years,  we  have  realized  that  waivers  to  the  really  significant  things 
that  prevent  true  reinvention  are  simply  too  difficult  for  the  system  to  deal 
with.  If  wholesale  waiving  of  personnel  and  funding  regulations  is 
needed,  we  must  achieve  it  by  other  means,  such  as  through  legislation. 

Nevertheless,  it  has  also  become  clear  that  the  use  of  waivers  can,  indeed, 
be  a  very  useful  enabling  device  for  a  larger  reinvention  initiative.  In  ARL's 
case,  there  are  a  myriad  minor  rules,  regulations,  forms,  and  various  other 
annoyances  that,  if  done  away  with,  could  save  the  support  staff,  in  par¬ 
ticular,  many  hours  of  useless  effort  that  could  be  better  used  in  support  of 
the  technical  staff.  It  was  with  this  philosophy  that  we  entered  into  the 
waiver  hunt  with  great  zeal. 
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4.2  Implementation:  The  "Old"  System 

Lacking  any  sort  of  guidance  as  to  how  waivers  could  be  obtained,  in 
March  1995  we  dived  into  the  process  by  devising  our  own  waiver  request 
format  and  preparing  a  set  of  55  waiver  requests,  which  we  bound  into  a 
volume  and  started  up  the  approval  chain,  accompanied  by  a  briefing  de¬ 
livered  to  several  members  of  the  AMC  and  Army  senior  leadership.  Of 
the  55  requests,  most  addressed  six  functional  areas  (human  resources 
management,  fiscal  resources  management,  procurement,  information 
resources  management,  capital  assets  management,  and  logistics),  along 
with  a  few  "miscellaneous."  The  requests  were  targeted  at  the  agencies  re¬ 
sponsible  for  the  regulations  that  we  desired  to  waive.  These  included 
AMC  headquarters  (our  parent  command),  Department  of  the  Army  head¬ 
quarters,  OSD,  the  Office  of  Personnel  Management  (OPM),  the  Office  of 
Management  and  Budget  (OMB),  the  Defense  Logistics  Agency  (DLA),  the 
Government  Printing  Office  (GPO),  and,  for  those  regulations  that  we  be¬ 
lieved  would  require  statutory  relief.  Congress. 

Waivers  were  solicited  from  the  members  of  the  ARL  staff,  particularly  the 
functional  support  staff.  Although  it  took  a  little  convincing  to  make  them 
believe  that  this  was  a  real  opportunity  for  relief,  many  suggestions  did 
come  in.  Each  was  vetted  by  management  and  by  our  Chief  Counsel  for 
appropriateness,  completeness,  utility,  and  "genealogy"  (that  is,  respon¬ 
sible  agency).  The  genealogy  of  a  waiver  often  turned  out  to  be  particu¬ 
larly  vexing,  sometimes  standing  in  the  critical  approval  path.  A  member 
of  the  support  staff  would  cite  a  regulation  that  was  believed  to  be  the 
cause  of  some  problem  or  inefficiency;  upon  investigation,  however,  it 
often  turned  out  that  the  particular  regulation  was  either  a  derivative  of  a 
higher  level  regulation  (which  is  what  we  really  needed  relief  from),  or 
was  being  implemented  more  stringently  than  required  at  a  lower  level, 
sometimes  even  internally  at  ARL.  Thus,  the  legal  review  became 
extremely  important.  Even  with  this  precaution,  several  waiver  requests 
were  returned  for  having  incorrect  cites. 

The  book  of  waivers  was  first  briefed  to  the  Deputy  Commander  of  AMC, 
who  took  action  on  the  spot  to  approve  five  of  the  six  AMC  waivers.  He 
also  gave  us  permission  to  take  the  book  forward  to  the  next  level.  We  next 
briefed  the  Assistant  Secretary  of  the  Army  (Research,  Development  and 
Acquisition),  who  was  enthusiastic  about  our  initiative  and  directed  his 
staff  to  take  charge  of  moving  the  rest  of  our  waivers  through  the  system  as 
quickly  as  possible. 

The  results  of  this  first  effort  were  as  follows: 

Approved  9 

Denied  6 

Pending  9  (most  of  these  were  never  heard  of  again) 

Returned  for  rework  6 

Withdrawn  18  (most  of  these  dealt  with  personnel  issues 

that  were  subsumed  into  the  personnel 
demo  program;  see  sect.  6) 


No  response 
Total 


7 

55 
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4.3  Implementation:  The  "New"  System 

While  we  were  pressing  forward  with  our  book  of  waiver  requests,  several 
new  processes  were  being  instituted  to  bring  some  order  to  the  system.  On 
23  May  1995,  the  Secretary  of  Defense  signed  a  letter  giving  authority  for 
any  organization  designated  a  Reinvention  Laboratory  or  Reinvention 
Center  to  request  waivers  of  any  nonstatutory  DoD  rules,  policies,  or  regu¬ 
lations.  There  were  a  few  restrictions  on  what  could  be  waived,  but  basi¬ 
cally  the  policy  was  that  if  the  request  were  not  personally  denied  within  30 
days  by  the  Secretary  or  the  Deputy  Secretary,  it  would  take  effect.  This 
policy  letter  was  prepared  by  the  Defense  Performance  Review  office. 
(Soon  after,  it  was  disestablished  and  its  functions  assigned  to  the  DoD 
Comptroller.)  Thus,  no  review  process  was  established  and  no  reviewing 
responsibility  determined.)  This  worked  out  very  well  for  the  field  agen¬ 
cies  who  managed  to  send  waiver  requests  into  the  system,  since  30  days 
would  pass  with  no  word  from  OSD,  and  the  waivers  would  take  effect. 

Unfortunately,  this  lapse  of  control  was  eventually  discovered,  and  on 
12  February  1996,  the  Deputy  Secretary  of  Defense  issued  a  new  directive 
"improving"  the  waiver  authority  process  by  extending  the  waiting  period 
to  90  days  and  making  it  considerably  more  complex.  This  modified  DoD 
system  envisioned  various  points  of  contact,  a  process  for  reviews, 
reclamas  against  denials,  postings  on  bulletin  boards,  etc.  However,  none' 
of  it  ever  happened;  the  DoD  waiver  system  was  completely  broken. 

A  new  group  of  energetic  OSD  staffers  then  took  hold  of  the  problem,  and 
on  2  April  1997  a  memo  signed  by  the  DoD  Comptroller  introduced  a  new 
and  truly  improved  system.  We  are  now  waiting  to  see  how  well  it  works, 
but  hopes  are  high. 

Meanwhile,  on  14  August  1995,  the  Secretary  of  the  Army  followed  the 
original  DoD  lead  and  took  it  a  step  further.  He  delegated  responsibility 
for  approving  waivers  of  Army  regulations  and  policies  to  the  local 
organization's  commander  or  director,  with  the  only  requirement  being  the 
notification  of  higher  headquarters  (this  authority  was  subject  to  the  same 
restrictions  as  the  original  DoD  policy:  i.e.,  only  nonstatutory  regulations 
could  be  waived,  waivers  could  not  affect  the  rights  of  employees  or  of 
bargaining  units,  and  waivers  could  not  augment  any  organization's  re¬ 
sources).  This  policy  has  been  the  most  flexible  method  for  dealing  with 
waivers. 

However,  on  4  November  1997,  DA  headquarters  issued  a  memorandum 
promulgating  a  revised  policy,  which  claims  to  "facilitate  the  process"  and 
"assist  in  the  resolution  of  differences  between  the  requesting  reinvention 
center  or  laboratory  and  the  HQDA  staff  principal."  Since  there  had  been 
few,  if  any,  such  "differences"  that  needed  resolution,  this  letter  set  off  a 
firestorm  of  complaints,  which  went  directly  to  the  Army  Chief  of  Staff. 
The  unofficial  feedback  is  that  the  headquarters  staff  should  not  interpose 
themselves  in  the  reinvention  process.  We  are  all  awaiting  the  official  re¬ 
sponse  from  the  Chief's  office. 


22 


At  the  AMC  level,  on  26  September  1995,  the  AMC  Commanding  General 
gave  local  commanders  and  directors  the  authority  to  waive  AMC  regula¬ 
tions  and  policies  (again  subject  to  the  same  restrictions)  by  submitting  re¬ 
quests  to  AMC  headquarters  and  waiting  30  days  for  denials,  failing 
which,  the  waivers  become  effective. 


4.4  Current  Status 

To  date,  ARL  has  submitted  a  total  of  117  waivers  to  be  processed  under 
both  the  "old"  and  "new"  systems.  Of  these,  41  have  been  approved.  These 
have  been  tallied  in  table  2  in  two  ways:  by  functional  area  and  by  granting 
authority. 


4.5  Assessment  of  Waivers  Program 


Table  2.  Status  of 
waiver  requests,  by 
type  and  authority. 


A  cursory  examination  of  table  2  makes  a  couple  of  points  obvious.  The 
most  fruitful  areas  have  been  logistics  and  the  management  of  our 
facilities.  This  is  not  surprising,  since  these  areas  are  not  the  lightning  rods 
that  personnel  or  fiscal  resource  management  are;  gaining  such  waivers 
does  not  generate  the  scrutiny  or  opposition  that  might  arise  in  more  sen¬ 
sitive  areas.  By  the  same  token,  gaining  a  waiver  in  these  areas  is  not  the 
seminal  event  that  determines  an  organization's  future.  Nevertheless,  the 
opportunities  here  are  not  trivial.  For  instance,  a  set  of  logistics  waivers  al¬ 
lowed  us  to  reduce  the  number  of  items  on  our  property  books  by  fully  25 
percent:  we  took  100,000  items  off  the  books.  This  led  to  substantial  sav¬ 
ings  in  the  nuisance  workload  of  both  support  and  technical  personnel 


Category 

Waiver  status 

Total 

Approved 

Pending 

Disap¬ 

proved 

With¬ 

drawn 

waivers 

Functional  area 

Personnel 

2 

— 

2 

13 

17 

Resource  management 

2 

— 

3 

13 

18 

Procurement 

4 

— 

2 

9 

15 

Info,  resource  mgmt 

— 

— 

1 

1 

2 

Capital  assets 

5 

— 

2 

1 

8 

Logistics 

17 

3 

— 

11 

31 

Miscellaneous 

11 

— 

2 

13 

26 

Total 

41 

3 

12 

61 

117 

Granting  authority 

AMC 

7 

1 

1 

7 

16 

DA 

24 

2 

6 

26 

58 

DoD 

9 

— 

2 

16 

27 

OPM 

— 

— 

— 

7 

7 

OMB 

— 

— 

— 

1 

1 

DLA 

— 

— 

1 

— 

1 

GPO 

— 

— 

1 

— 

1 

Statutory 

i 

— 

1 

4 

6 

Total 

41 

3 

12 

61 

117 

23 


Table  3.  Savings  due 
to  34  waivers. 


during  annual  property  inventories.  Likewise,  although  many  of  the  other 
approved  waivers  yield  only  small  savings  of  support  workload,  these  add 
up.  At  the  very  least,  the  waivers  simplify  our  lives  by  removing  many  of 
the  annoying  and  seemingly  mindless  chores  that  clutter  our  days.  This 
saves  precious  time  for  concentrating  on  more  important,  mission-related 
activities. 

The  abundance  of  DA  waivers  could  also  have  been  predicted,  since  the 
DA  waiver  system  has  been  the  most  flexible  in  terms  of  delegation  of  au¬ 
thority.  In  fact,  it  is  the  very  essence  of  the  NPR  principle  to  empower  em¬ 
ployees  in  this  case,  senior  leadership  at  the  organization  level.  In  con¬ 
trast,  the  fact  that  we  have  had  any  success  at  all  with  DoD  waivers  is  only 
because  we  got  our  applications  in  before  that  system  fell  apart. 

A  large  number  of  waiver  requests  were  withdrawn  for  two  principal  rea¬ 
sons.  First,  several  issues  were  addressed  by  other  means.  For  example, 
most  of  the  personnel  waiver  requests  were  withdrawn  so  that  they  could 
be  included  in  the  alternative  personnel  system  demonstration  (discussed 
in  sect.  6).  Likewise,  other  requests  were  withdrawn  because  they  were 
overtaken  by  events,  as  other  activities  in  which  ARL  has  been  involved 
took  up  their  cause.  Most  notable  among  these  activities  is  the  Laboratory 
Quality  Improvement  Program  (LQIP)  (also  discussed  in  sect.  6). 

Second,  some  of  the  requests  were  withdrawn  or  disapproved  simply 
because  we  did  not  do  sufficient  research  or  preparation,  or  frankly,  we 
overreached. 


We  recently  conducted  a  formal  audit  of  our  waiver  program  to  determine 
its  specific  impact  on  funding  and  manpower  savings,  if  any.  The  ARL  In¬ 
ternal  Review  and  Audit  Compliance  Office  reviewed  34  of  our  waviers. 
The  results  which  are  shown  in  table  3,  revealed  only  a  few  small  dollar 
savings  (some  of  them  only  one-time  events).  Neither  have  we  found  labor 
savings  of  the  sort  that  could  be  used  to  take  FTEs  in  bulk  off  our  rolls.  We 
have  found  some  labor  savings  distributed  throughout  the  lab.  We  have 
been  able  to  eliminate  many  nuisance  requirements,  shaving  a  few  hours 
off  here  or  a  couple  of  days  off  there  from  a  variety  of  processes,  mostly  in 
logistics  and  procurement.  Most  of  these  result  in  time  savings  for  the  tech¬ 
nical  staff,  in  that  they  can  obtain  the  needed  support  services  with  shorter 
waiting  times.  We  have  also  been  able  to  gain  relief  from  regulations  that 
were  causing  morale  problems  for  staff  members  in  particular  circum¬ 
stances,  such  as  those  on  long-term  rotations  to  our  FedLab  partners7  re¬ 
search  facilities,  or  those  caught  up  in  BRAC-related  relocations.  We  have 
also  been  able  to  avoid  doing  silly  things,  like  putting  inventory  control 
bar  code  stickers  on  relocatable  buildings  (a  projected  savings  of  0.06 
work-days  per  year!). 


One  time 

Annual 

Cost 

$96K 

$230 

Labor 

313  mandays 

1006  mandays 

Process  time 

475  days 

2127  days 
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An  issue  of  immediate  concern  is  the  requirement  that  waivers  were  only 
to  be  for  two  years.  That  means  that  most  of  the  waivers  granted,  not  only 
to  ARL  but  to  the  other  LQIP  labs  as  well,  are  running  out.  The  expiration 
of  these  has  temporarily  been  put  on  hold  by  all  three  services  and  OSD 
while  possible  approaches  are  being  considered.  For  ARL,  even  though  the 
waivers  were  less  than  an  overwhelming  success,  nevertheless  having  to 
re-enter  those  100,000  pieces  of  waived  property  on  hand  receipts  would 
cause  chaos.  Thus,  we  are  working  through  the  LQIP  for  a  more  rational 
and  permanent  solution  to  this  problem. 

Despite  the  lack  of  dramatic  cost  and  time  savings  that  people  seem  to  ex¬ 
pect  from  these  types  of  waivers,  we  intend  to  continue  participating  in  the 
program  to  the  greatest  extent  possible  and  to  shape  it  to  more  strongly 
support  the  FedLab  and  Open  Lab  initiatives. 
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5.  Business  Process  Reengineering  (BPR) 

The  waivers  obtained  through  our  participation  in  the  NPR  tended  to 
counter  the  burdens  of  regulations  and  policies  externally  imposed  by 
higher  headquarters.  However,  upon  examination,  we  found  numerous 
internally  generated  processes  that  were  also  creating  time  and  manpower 
burdens  on  the  technical  staff.  Through  the  techniques  of  Business  Process 
Reengineering  (BPR),  we  examined  many  of  our  own  processes.  Through 
this  initiative,  we  identified  numerous  opportunities  to  realize  efficiencies, 
which  could  facilitate  the  implementation  of  FedLab,  as  well  as  enable 
other  internal  programs  to  continue  despite  the  continuing  drawdown  of 
support  personnel. 

5.1  What  is  BPR? 

Business  Process  Reengineering  is  a  fundamental  rethinking  and  basic 
redesign  of  an  entire  business  system,  including  processes,  jobs,  organiza¬ 
tional  structures,  and  management  systems.  It  emphasizes  the  use  of  auto¬ 
mation  to  improve  process  performance.  By  radically  and  dramatically 
changing  business  processes,  BPR  aims  to  increase  process  efficiency,  effec¬ 
tiveness,  productivity,  and  quality,  benefiting  both  the  organization  and  its 
customers. 

When  an  organization  experiences  personnel  drawdown  (as  ARL  is),  it  is 
necessary  to  preserve  as  much  as  possible  of  the  workforce  responsible  for 
"putting  out  the  product."  This  was  especially  so  as  we  began  our  FedLab 
reinvention  efforts,  since  the  FedLab  was  predicated  on  maximizing  the 
leveraging  that  our  technical  staff  could  do  with  respect  to  our  new  exter¬ 
nal  partners.  Thus,  the  support  staff  must  suffer  most  of  the  cuts.  Cutting 
the  support  staff,  however,  risks  reducing  the  technical  staff's  support  to 
the  point  where  the  overall  effectiveness  of  the  organization  suffers.  The 
solution  is  to  find  ways  to  perform  the  support  functions  more  efficiently 
with  fewer  people.  Although  some  of  this  increased  efficiency  might  come 
from  waiving  external  rules  and  regulations  (as  discussed  in  the  previous 
section),  ARL  also  recognized  the  need  to  examine  internal  policies  and 
procedures  that  might  be  outdated,  outmoded,  irrelevant,  or  inefficient. 
The  methodology  chosen  was  BPR,  and  it  was  implemented  through  a 
Business  Process  Reengineering  Office  established  at  ARL  on  a  temporary 
basis. 

The  purpose  of  the  BPR  Office  was  to  support  ARL's  efforts  to  attain  a  cor¬ 
porate  information  management  environment  in  which  personnel,  finan¬ 
cial,  accounting,  acquisition,  and  material  functions  operate  uniformly 
throughout  ARL.  The  assumption  was  that  standardizing  support  proc¬ 
esses,  along  with  streamlining  them,  would  increase  overall  efficiency.  This 
standardization  will  also  have  the  secondary  benefit  of  helping  to  unify  the 
seven  formerly  separate  laboratories  into  a  more  cohesive  entity. 

Because  ARL  was  formed  from  several  pre-existing  organizations  (see  the 
discussion  of  ARL  s  history  in  sect.  1),  many  business  processes  were  per¬ 
formed  differently  at  ARL's  three  major  sites:  the  Adelphi  Laboratory  Cen- 
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ter  (ALC)  in  Adelphi,  MD;  Aberdeen  Proving  Ground  (APG)  in  Aberdeen, 
MD;  and  White  Sands  Missile  Range  (WSMR)  in  White  Sands,  NM.  (Since 
the  ARL  units  colocated  with  the  NASA  Langley  and  Lewis  Laboratories 
rely  on  these  NASA  organizations  for  their  support,  they  were  not  in¬ 
cluded  in  this  BPR  initiative.)  The  expected  outcome  of  reengineering  was 
that  differences  from  site  to  site  would  be  the  exception  rather  than  the 
rule.  The  BPR  Office  examined  business  processes  to  determine  whether  or 
not  they  were  required,  and  if  they  were,  the  most  efficient  way  to  carry 
them  out.  BPR  also  identifies  areas  where  information  technology  can  im¬ 
prove  activities  within  a  given  process.  The  overall  objective  is  to  transition 
the  business  processes  from  site-specific  processes  to  consistent  ARL-wide 
corporate  processes,  by  using  better  business  practices  and  automation 
support.  The  BPR  Office  limited  its  efforts  to  the  business  functions  in  the 
Office  of  the  ARL  Chief  of  Staff,  the  support  arm  of  the  laboratory.  The 
scope  of  the  BPR  project  was  limited  to  the  major  support  activities:  hu¬ 
man  resources,  financial  management,  logistics,  and  procurement.  Table  4 
shows  the  business  units  examined  by  the  BPR  Office  and  their  relation¬ 
ship  to  the  four  functional  areas. 

The  expectation  was  that  BPR  would  lead  to  streamlined,  efficient,  and 
cost-effective  business  processes.  It  would  identify  the  optimal  integrated 
information  system  to  be  developed  to  support  the  business  needs,  and  it 
should  make  visible  the  strengths  and  weaknesses  in  the  organization's 
business  processes. 

5.2  BPR  Implementation 

BPR  is  a  comparative  analysis  of  where  we  are  today  with  where  we 
should  be  in  the  future.  It  determines  where  we  are  today  through  the 
development  of  a  "Type  I  Enterprise  Model."  This  model,  a  high-level 
overview  of  the  organization's  structure  and  functionality,  documents 
where  business  functions  are  performed  in  the  current  environment.  It 
serves  to  highlight  functional  duplication.  Based  on  a  series  of  interviews 
with  senior  management,  functional  heads,  and  key  personnel  in  both  the 
S&E  and  the  operational  units,  high-level  finance,  resource,  and  infrastruc¬ 
ture  models  were  constructed. 


Table  4.  Business  units  by  functional  area. 


Human 

resources 

Financial 

management 

Logistics 

Procurement 

Mixed 

functions 

Human  Resource 

Resource  Mgt 

Logistics  Mgmt 

Procurement 

Legal  Office 

Mgmt  Div 

Div 

Div 

Div 

Technology  Transfer 

Equal 

Program  and 

Infrastructure 

Small  and 

Office 

Employment 

Opportunity 

Office 

Manpower  and 
Force  Analysis 
Branch 

Budget  Office 

Mgmt  Office 
Public  Works  Div 
Risk  Mgmt  Div 

Disadvantaged 
Business 
Utilization  Ofc 
Competition 
Advocate 

Law  Enforcement 
and  Physical 

Security  Div 
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The  next  step  was  to  formulate  the  "Type  I  Best  Practices  Model,"  as  deter¬ 
mined  through  benchmarking  against  organizations  considered  to  be  "best 
in  class.  The  Best  Practices  Model  documents  where  business  functions 
should  be  performed  and  projects  functional  efficiencies  and  improved 
internal  controls. 

In  order  to  formulate  the  Best  Practices  Model,  the  BPR  team  conducted 
benchmark  interviews  with  Malcolm  Baldrige  and  Presidential  Quality 
Awards  winners  and  finalists,  such  as  IRS  Ogden,  IBM  Rochester,  Eastman 
Chemical,  NASA  Lewis,  U.S.  Patent  and  Trademark  Office,  Xerox, 
Selectron,  and  Motorola.  Interviews  were  conducted  with  a  cross  section  of 
key  personnel  at  each  organization.  Special  emphasis  was  given  to  compa¬ 
rable  functions  at  ARL,  in  particular,  purchasing  and  receiving,  financial 
management  and  control,  human  resources,  and  information  resources. 

Finally,  the  BPR  team  performed  a  gap  analysis  by  comparing  the  Type  I 
Best  Practices  Model  to  the  Type  I  Enterprise  Model.  This  comparison 
identified  opportunities  to  eliminate  redundant  functions  and  to  realign 
functions  in  order  to  improve  processing  or  internal  business  and  quality 
controls.  Improvement  opportunities  were  based  on  four  primary  criteria: 

•  identified  standard  best  industry  practices, 

•  estimated  improvements  in  processing  time, 

•  estimated  improvements  in  customer  service,  and 

•  reduced  cost. 

5.3  Findings  of  Type  I  Analysis 

The  BPR  Office  predicted  that,  as  a  result  of  implementing  the 
reengineering  opportunities,  ARL  support  staff  would  be  able  to  deliver  an 
improved  level  of  service  to  the  technical  staff  without  redundant  func¬ 
tions  or  unnecessary  costs;  internal  business  and  quality  controls  would  be 
maintained  or  improved;  and  all  primary  functions  would  be  retained. 

Examining  best  practice  trends  revealed  several  important  opportunities 
for  the  alignment  of  functions  and  processes  relative  to  the  total  ARL  orga¬ 
nization.  For  instance,  by  retaining  a  backbone  of  corporate  administration 
and  support  functions  organized  into  expert  cross-functional  support  ser¬ 
vices  teams,  ARL  could  eliminate  redundancies  and  improve  service.  In 
order  to  promote  customer  service  and  advance  the  ability  of  the  support 
teams  to  understand  the  technical  mission  of  the  units  they  support,  the 
supported  units  should  evaluate  the  performance  of  the  teams.  Also,  pro¬ 
motions,  awards,  and  professional  development  of  the  support  team 
should  be  linked  to  the  needs  and  evaluations  of  supported  units. 

5.4  Type  II  Reengineering 

With  the  completion  of  the  macro-evaluation,  the  next  step  is  a  more  in- 
depth  examination  of  processes,  called  "Type  II  Reengineering."  Here  the 
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BPR  Office  used  a  software  package  to  model  dozens  of  individual  proc¬ 
esses  in  the  identified  functional  areas.  This  modeling  effort  sought  to  un¬ 
derstand  all  the  steps  involved  in  these  processes,  and  then  identify  for 
elimination  those  that  do  not  add  value.  The  model  examines  the  inputs, 
outputs,  controls,  and  mechanisms  for  each  process  and  lays  them  out  (see 
fig.  4  for  these  features  of  the  process  "Issue  Travel  Orders").  First,  an  "as 
is"  model  is  formulated,  representing  the  current  state  of  the  process  being 
modeled,  without  any  specific  improvements  included.  At  this  time,  cost¬ 
ing  information  is  obtained  for  each  activity  within  the  model.  (The  "as  is" 
model  was  usually  different  at  ALC,  APG,  and  WSMR,  and  the  modeling 
was  conducted  separately  at  each  site.)  Then,  a  "to  be"  model  is  formu¬ 
lated,  representing  the  proposed  future  state  as  a  result  of  the  redesigned 
process.  The  basic  "to  be"  model  is  the  same  for  all  sites,  representing  one 
unified  ARL  approach  with  standard  business  practices  across  ALC,  APG, 
and  WSMR.  Costing  information  is  also  estimated  for  the  activities  in  the 
reengineered  "to  be"  model,  allowing  comparisons  to  be  made.  The  differ¬ 
ences  in  cost  and  the  quantity  of  full-time  work-years  between  the  "to  be" 
and  the  "as  is"  models  are  the  potential  savings  that  are  projected  to  be 
achievable  upon  full  implementation  of  the  recommendations. 

Surveys  of  leaders  in  functional  areas  yielded  a  comprehensive  list  of  proc¬ 
esses  nominated  for  this  Type  II  examination.  In  the  end,  the  processes 
were  pared  down  to  the  following  list  for  the  detailed  Type  II 
Reengineering,  and  detailed  sets  of  recommendations  were  made  to  im¬ 
prove  each  process: 

•  acquisition  plan  of  execution 

•  ammunition  handling 

•  awards 

•  complaints/  grievances 

•  cooperative  agreements 


29 


•  cost/ price  analysis 

•  formal  contracts 

•  funds  control  and  distribution 

•  General  and  Administrative  (G&A)  overhead  rate  calculation/application 

•  indirect  overhead  rate  calculation /distribution 

•  internal  operating  budget 

•  lateral  transfers 

•  long-term  training 

•  obligation  forecasting 

•  personnel  actions 

•  risk  management  pharmacy  operations 

•  small  purchases 

•  storage  procedures 

•  technology  planning 

•  training 

•  travel  orders 

•  turn-ins 

•  unliquidated  obligation  reviews 


5.5  Current  Status,  Assessment,  and  Future  Directions 

ARL  senior  management  was  briefed  on  the  BPR  Office's  recommenda¬ 
tions  in  the  areas  listed  above.  It  is  projected  that  if  these  recommendations 
are  fully  implemented,  the  reengineering  will  save  over  70  FTEs  in  man¬ 
power  per  year  across  ARL,  which  equates  to  approximately  $3.5M  in  sav¬ 
ings  per  year.  For  example,  to  produce  the  standard  Request  for  Personnel 
Action,  SF-52,  BPR  analysis  showed  how  to  reduce  the  current  31-step 
process  to  17  steps,  with  a  projected  annual  savings  of  9.25  FTEs  and 
$460K. 


Table  5.  Annual 
projected  savings 
identified  through 
reengineering  at 
ARL. 


Such  savings  can  be  realized  once  full  implementation  is  complete.  Table  5 
shows  these  savings  by  groups  of  processes  within  each  functional  area.  It 
is  important  to  note  that  all  FTE  and  dollar  savings  do  not  necessarily 
come  from  one  particular  functional  area,  but  may  be  spread  across  mul¬ 
tiple  areas,  including  the  technical  directorates. 

ARL  is  moving  to  put  these  reengineered  processes  into  place.  Implemen¬ 
tation  has  already  begun  for  personnel  actions,  training  requests,  awards, 
travel  orders,  and  small  purchases.  Two  cross-functional  teams,  which 
were  designed  to  support  the  recommendations  of  the  reengineering  ef¬ 
forts,  were  formed  and  began  working  in  1996.  These  self-managed  teams. 


Process 

areas 

Annual 
saving  ($) 

Annual 

FTE 

savings 

Financial  mgmt 

502,383 

9.5 

Human  resources 

790,629 

17.1 

Logistics 

473,877 

8.3 

Procurement 

1,635,923 

38.3 

Total 

3,402,812 

73.2 
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made  up  of  people  in  all  four  functional  areas  (plus  a  team  leader),  were 
supposed  to  provide  "one-stop  shopping"  for  the  technical  staff  to  obtain 
support  functions.  The  teams  had  to  struggle  with  the  usual  obstacles  in¬ 
herent  in  changing  the  old  ways  of  doing  business.  They  had  to  define 
their  identity  and  roles,  and  deal  with  obstacles  placed  in  their  way  by  re¬ 
luctant  players  in  the  system.  Although  the  teams  had  begun  implement¬ 
ing  some  of  the  reengineering  recommendations,  additional  downsizing 
and  streamlining  of  the  Chief  of  Staff's  organization  resulted  in  the  teams 
being  disbanded  and  the  members  returned  to  their  prior  organizations. 

Has  reengineering  at  ARL  gone  as  expected?  The  answer  is  yes  and  no.  The 
processes  scheduled  for  reengineering  have  been  defined.  Although  some 
modifications  were  made  to  the  process  list,  the  BPR  effort  covered  sub¬ 
stantial  ground  in  the  four  functional  areas  of  human  resources,  financial 
management,  procurement,  and  logistics.  However,  full  implementation 
has  not  occurred,  mainly  because  of  the  lack  of  an  adequate  automated 
management  system. 

The  BPR  Office  completed  its  analysis  and  prepared  the  summary  docu¬ 
mentation,  including  all  the  recommended  process  changes.  The  report 
was  submitted  to  the  ARL  Director  and  the  Chief  of  Staff  for  their  consider¬ 
ation.  The  office  was  then  disbanded. 

Since  that  time,  automation  efforts  have  been  redoubled.  We  now  have  a 
completely  automated  personnel  action  and  position  description  system, 
which  was  brought  on  line  as  part  of  the  personnel  demonstration  pro¬ 
gram  (see  sect.  6.)  We  are  also  about  to  inaugurate  a  completely  automated 
purchasing  system  and  an  automated  travel  system.  In  addition,  an  on-line 
supply  system  has  been  running  very  successfully  for  about  a  year.  Thus, 
many  of  the  BPR  recommendations  have  indeed  been  enacted,  although 
not  always  in  the  way  they  were  originally  conceived. 
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6.  Laboratory  Quality  Improvement  Program  (LQIP) 
and  Alternative  Personnel  System  Demonstration 

In  the  search  for  mechanisms  to  enable  ARL  to  operate  more  efficiently,  we 
participate  in  the  Laboratory  Quality  Improvement  Program  (LQIP)  initia¬ 
tive.  Our  hope  was  to  obtain  additional  flexibilities  beyond  those  that  NPR 
and  BPR  could  deliver.  As  originally  conceived,  LQIP  promised  a  means  of 
achieving  relief  from  DoD-wide  constraints,  as  well  as  from  statutory  bar¬ 
riers  to  efficient  operation.  A  number  of  initiatives  were  undertaken  under 
the  aegis  of  the  DoD's  Director  of  Defense  Research  and  Engineering 
(DDR&E).  Most  were  less  than  completely  successful.  However,  the  one 
major  area  of  impact  for  ARL  is  the  alternative  personnel  system  demon¬ 
stration,  which  will  enable  us  to  manage  our  people  in  a  more  rational 
manner,  free  from  many  of  the  constraints  of  the  traditional  civil  service 
system.  This  is  important  for  any  research  organization,  but  will  be  espe¬ 
cially  useful  as  we  continue  to  reinvent  ourselves  under  the  FedLab 
concept. 

6.1  Roots  of  LQIP 

Over  the  past  30  or  more  years,  more  than  50  studies  of  the  Defense  labora¬ 
tory  system  have  repeatedly  identified  the  problems  of  excessive  and  inap¬ 
propriate  rules  and  regulations  and  of  external  micromanagement,  both  of 
which  reduce  productivity  and  effectiveness.  In  1987  the  Defense  Science 
Board,  during  a  summer  study  of  the  management  of  the  DoD  technology 
base,  expressed  great  concern  about  the  quality  of  the  in-house  laborato¬ 
ries  and  observed  that  the  inefficiencies  in  the  system  were  only  going  to 
get  worse,  thereby  reducing  the  effectiveness  of  the  labs  even  further.  The 
Board  recommended  to  the  DDR&E  that  a  program  of  improvement  and 
rejuvenation  be  instituted  within  the  DoD  labs  in  order  to  allow  them  to 
function  more  like  private  sector  labs.  Accordingly,  in  November  1989,  the 
laboratory  demonstration  (Lab  Demo)  program  was  established  by  the 
Deputy  Secretary  of  Defense.  Its  charter  was  to  identify  the  most  effective 
corrective  actions  for  the  long-recognized  systemic  problems  in  the  man¬ 
agement  of  technical  staffs  (and  the  various  support  functions  that  should 
empower  those  staffs),  and  obtain  the  authorization  to  implement  changes. 

Although  a  number  of  innovative  ideas  were  proposed  under  Lab  Demo, 
few  were  acted  upon.  In  1994,  the  new  DDR&E  decided  to  re-energize  the 
program,  the  LQIP  initiative  was  instituted.  Its  participants  were  the  origi¬ 
nal  Lab  Demo  labs.  Within  the  Army,  this  included,  in  addition  to  ARL,  the 
Research,  Development  and  Engineering  Center  of  the  Missile  Command 
(MRDEC),  the  Waterways  Experiment  Station  (WES)  of  the  Corps  of  Engi¬ 
neers,  and  the  laboratories  of  the  Medical  R&D  Command.  The  other  Ser¬ 
vices  were  also  represented  by  several  laboratories  each. 


6.2  Implementation  of  LQIP 

In  March  1994,  LQIP  was  designated  the  "DoD  Science  &  Technology 
Laboratory  Demonstration  Program  Reinvention  Laboratory"  by  the  NPR. 
Thus,  ARL  became  one  of  the  science  and  technology  (S&T)  reinvention 
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laboratories  under  this  umbrella.  LQIP  is  managed  by  an  Implementation 
Panel  that  comprises  the  Service  S&T  executives  (or  their  representatives), 
and  reports  to  the  Deputy  DDR&E  (LM&TT — Lab  Management  and  Tech¬ 
nology  Transfer).  Its  basic  purpose  is  to  take  on  problems  of  common  inter¬ 
est  to  the  labs  in  all  three  Services;  it  either  works  for  policy/ regulatory  re¬ 
lief,  or  prepares  legislative  packages  to  obtain  statutory  relief.  The  Imple¬ 
mentation  Panel  was  chaired  by  an  Army  representative  for  the  first  two 
years;  the  leadership  was  then  passed  to  the  Navy.  Under  the  panel  are  five 
subpanels: 

•  Legislative  Subpanel — seeking  legislative  relief  from  limitations  on  mili¬ 
tary  construction.  (This  panel  was  recently  redesignated  the  Facilities 
Subpanel.) 

•  Financial  Subpanel — seeking  to  develop  and  implement  a  new  financial 
system  that  will  enable  a  more  businesslike  approach  to  the  operations  of 
the  S&T  reinvention  labs. 

•  Personnel  Subpanel — managing  the  planning  process  for  the  proposed 
alternative  personnel  system  demonstration. 

•  Information  Infrastructure  Subpanel — assuring  that  DoD  S&Es  can  maxi¬ 
mize  their  use  of  the  information  tools  available. 

•  Waiver  Subpanel — addressing  several  issues  not  falling  under  the  purview 
of  the  other  subpanels. 

ARL  has  been  an  active  participant  in  all  these  subpanels,  except  the  Legis¬ 
lative/Facilities  Subpanel.  A  new  Procurement  Subpanel  has  just  been 
chartered,  for  which  ARL  is  supplying  the  Army  representative. 

6.3  Assessment  of  Current  Status  and  Future  Directions 

On  3-4  February  1997,  the  Deputy  DDR&E(LM&TT)  chaired  an  offsite 
meeting  of  the  Implementation  Panel  to  assess  the  accomplishments  of  the 
past  three  years  and  LQIP's  role  in  the  future.  The  basic  scorecard  for 
LQIP's  accomplishments  was  not  impressive: 

•  Resolve  Defense  Management  Review  Directive  (DMRD)  conflicts — A 
number  of  Directives  came  out  of  the  Defense  Management  Review  in 
1988  that,  while  appropriate  to  other  organizations  within  DoD,  were  com¬ 
pletely  inappropriate  for  R&D  organizations.  These  had  been  brought  to 
the  attention  of  then-Deputy  Secretary  of  Defense  Atwood  who,  in  October 
1992,  approved  in  principle  that  R&D  organizations  should  receive  partial 
or  total  relief  from  several  of  these  DMRDs.  This  would  have,  among  other 
things,  avoided  the  current  ongoing  regionalization  of  several  of  the  sup¬ 
port  functions.  However,  Atwood  failed  to  sign  the  order  for  this  before 
leaving  office.  Although  LQIP  tried  to  follow  up  on  the  several  DMRDs  in 
question,  it  failed  on  every  one. 

•  Raise  the  minor  Military  Construction  (MILCON)  budget — This  action  in¬ 
volved  a  legislative  package  to  raise  the  minor  construction  limit  from 
$300K  to  $1M,  and  the  unspecified  minor  construction  limit  to  $3M.  This 
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effort  was  successful  and  resulted  in  a  two-year  test  program  in  the  FY96 
Defense  Authorization  Bill.  The  test  period  is  running  out,  and  legislation 
is  being  sought  to  extend  it. 

•  Increase  small  purchase  threshold — An  effort  to  increase  the  small  pur¬ 
chase  limit  from  $25K  to  $100K  began  as  an  LQIP  initiative,  but  was  picked 
up  and  enacted  government-wide  by  the  FY95  Federal  Acquisition  Stream¬ 
lining  Act.  This  is  probably  the  most  significant  LQIP  accomplishment. 

•  Use  Cooperative  and  Other  Agreements— The  authority  to  use  Coopera¬ 
tive  Agreements  was  delegated  to  the  labs  as  a  result  of  this  LQIP  initia¬ 
tive.  This  authority  was  critically  important  as  the  enabling  waiver  that 
allowed  our  FedLab  reinvention  initiative.  The  authority  to  use  Other 
Agreements  is  still  at  the  OSD  level  and,  for  the  most  part,  is  being  used 
only  by  the  Defense  Advanced  Research  Projects  Agency  (DARPA). 

•  Allow  labs  to  manage  to  budget  versus  manage  to  budget  and  to  FTEs — 
The  thrust  of  this  waiver  was  to  allow  laboratories  to  manage  their  person¬ 
nel  resources  strictly  based  on  what  types  and  numbers  of  people  their 
budgets  would  allow.  This  is  in  contrast  to  the  current  practice  of  a  lab 
director  being  constrained  not  only  by  the  budget,  but  also  by  a  collection 
of  nonbudgetary  limits  or  caps  on  numbers  of  people,  kinds  of  people, 
grade  levels  of  staff  members,  and  so  forth.  Although  the  idea  of  managing 
to  budget  only  was  initially  stressed  by  the  NPR,  the  concept  never  got  off 
the  ground  for  the  labs— this,  despite  the  fact  that  at  the  Second  National 
Reinvention  Revolution  Conference  held  on  7-9  April  1997,  the  Vice  Presi¬ 
dent  stated  in  direct  and  unmistakable  language  that  managing  by  FTE 
levels  was  no  longer  to  be  practiced. 

•  Streamline  R&D  procurement  procedures— The  improvement  was  ap¬ 
proved  by  the  Defense  Acquisition  Review  Council  for  a  test,  but  it  has 
seen  only  limited  use. 

•  Establish  Alternative  Personnel  System— The  FY95  Defense  Authorization 
Act  authorized  the  use  of  Alternative  Personnel  Systems.  (See  sect.  6.4  for  a 
discussion  of  progress.) 

With  this  track  record,  LQIP  seems  to  have  been  something  less  than  a 
blazing  success.  However,  the  members  decided  to  continue  the  process,  if 
for  no  other  reason  than  to  provide  a  forum  for  senior  S&T  managers  from 
the  lab  system  to  work  together  on  common  problems,  as  well  as  to  pro¬ 
vide  assistance  to  the  Deputy  DDR&E  (LM&TT)  in  his  efforts  to  be  an  ad¬ 
vocate  for  our  interests. 


6.4  Alternative  Personnel  System  Demonstration 

The  effectiveness  of  the  Defense  laboratories  has  been  studied  by  a  variety 
of  blue  ribbon  panels,  committees,  and  commissions  for  at  least  four 
decades.  Every  one  of  these  studies  has  concluded  that  the  principal  prob¬ 
lem  that  the  labs  have  in  fully  exploiting  their  capabilities  is  the  civil  ser¬ 
vice  personnel  system.  The  system  introduces  delays  in  performing  even 
the  simplest  personnel  actions.  It  ties  the  laboratory  director  's  hands  in 
such  crucial  areas  as  hiring,  promoting,  disciplining,  training,  restructur- 
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Figure  5.  Career  paths 
and  pay  bands. 


ing,  etc.  This  system  is  agreed  to  be  the  single  most  important  barrier  pre¬ 
venting  in-house  government  labs  from  functioning  with  the  same  effi¬ 
ciency  as  their  sister  labs  in  the  private  sector,  and  this  argument  has  been 
frequently  used  as  a  rationale  for  privatizing  the  labs  or  turning  them  into 
GOCO  (government-owned,  contractor-operated)  operations. 

For  ARL,  this  problem  was  particularly  vexing,  since  as  the  Army's  "cor¬ 
porate"  laboratory,  we  would  be  compared  to  world-class  laboratories  like 
the  legendary  Bell  Labs,  which  does  not  suffer  from  such  crushing  con¬ 
straints  on  the  way  it  manages  its  personnel.  Furthermore,  if  we  were  to  be 
able  to  implement  certain  planned  aspects  of  FedLab,  we  would  very  defi¬ 
nitely  need  relief  from  the  existing  system. 

The  opportunity  to  address  this  long-standing  problem  came  when  the 
FY95  Defense  Authorization  Bill,  Section  342,  empowered  the  Secretary  of 
Defense  to  make  the  15-year-old  "China  Lake"  personnel  demonstration 
experiment  permanent,  and  to  expand  it  to  the  other  S&T  reinvention  labo¬ 
ratories.  This  authority  was  delegated  to  the  DDR&E  to  implement,  and 
LQIP  was  the  mechanism  chosen  to  carry  out  this  implementation.  The 
LQIP  personnel  subpanel  has  been  guiding  and  coordinating  the  prepara¬ 
tion  of  the  personnel  demo  proposals  from  the  S&T  reinvention  labs  in  all 
three  services.  Although  the  plans  vary  in  detail,  they  are  all  aimed  at  al¬ 
lowing  labs  to  hire  and  retain  the  best  and  the  brightest  from  the  scientific 
and  engineering  communities,  and  to  deal  with  human  resources  more  in 
the  manner  that  the  private  sector  does. 

The  ARL  proposal  was  submitted  along  with  those  of  the  other  Army  S&T 
reinvention  labs.  The  features  of  the  ARL  proposal  include 

speed  and  flexibility  in  recruitment,  hiring,  and  placement, 

simplified  separation, 

pay  for  performance, 

simplified  classification  (pay  banding,  including  a  new  senior-level  band; 
see  fig.  5), 

improved  staff  development  program,  and 
a  simplified  rewards  system. 
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The  goals  of  the  program  are  to 

•  improve  hiring  and  allow  ARL  to  effectively  compete  for  high-quality 
personnel, 

•  motivate  and  retain  high-quality  staff, 

•  strengthen  the  manager's  role, 

•  increase  the  personnel  system's  efficiency, 

•  revitalize  the  research  intellect  by  facilitating  personnel  training,  acquisi¬ 
tion,  and  staff  mobility, 

•  remain  budget  neutral  by  managing  to  in-house  budget, 

•  create  a  model  that  can  be  exported  to  other  agencies,  and,  of  central 
importance, 

•  facilitate  the  FedLab  concept  of  operations  through  simplified  and  more 
flexible  staff  management. 

The  initial  Army  proposal  (dated  11  April  1995)  went  through  numerous 
iterations,  and  three  joint  AMC-DA-DoD-OPM  reviews.  The  reviews  iden¬ 
tified  problem  areas  in  the  proposal  and  provided  potential  solutions  and 
additional  innovations  not  previously  identified.  Along  the  way,  several 
important  features  were  designated  "Protected  Corporate  Interests"  by 
DoD  and  removed  from  the  proposal.  These  included  relief  from 

•  high-grade  controls, 

•  the  priority  placement  program, 

•  PBG  (Program  Budget  Guidance) /FTE  controls,  and 

•  regionalization  of  the  DoD  Civilian  Personnel  Offices  (CPO). 

Although  relief  was  sought  by  the  DDR&E  personally,  none  was  granted. 
Even  the  intervention  of  the  NPR  staff  at  the  direction  of  the  Vice  President 
was  not  successful.  With  these  critical  features  removed,  the  plan  was 
approved  by  the  Army  and  forwarded  to  DoD  on  13  December  1996.  After 
DoD  and  OPM  approval,  the  proposal  was  published  in  the  12  March  1997 
Federal  Register,  and  a  copy  provided  to  each  affected  employee.  The 
required  public  hearings  were  held  on  17-18  April  1997. 

The  proposal  was  approved  by  DoD,  concurred  in  by  OPM,  and  published 
in  final  form  in  the  Federal  Register  on  4  March  1998.  The  resulting 
Alternative  Personnel  System  Demonstration  plan  was  implemented  at 
ARL  on  7  June  1998. 

The  work  force  was  involved  in  the  planning  through  a  Staff  Members 
Committee,  which  facilitated  communication  of  the  various  aspects  of  the 
proposed  demo  and  assured  the  work  force  that  there  would  be  no  abridg¬ 
ment  of  their  rights.  The  plan  was  also  coordinated  with  the  unions  and 
has  received  their  approval  and  buy-in  to  the  process. 
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ARL  established  a  program  management  office  and  an  executive  steering 
committee  to  manage  the  process  and  interface  with  the  Army  and  LQIP 
officials  involved  with  the  overall  implementation  of  the  Congressional  di¬ 
rective.  Now  that  the  demo  is  successfully  under  way,  we  are  hopeful  that 
at  least  some  of  the  barriers  to  personnel  management  in  the  DoD  labora¬ 
tory  system  will  be  removed,  although  the  more  complete  relief  we  had 
sought  will  have  to  wait  for  further  initiatives. 

On  24  February  1998,  Principal  Deputy  Director  of  NPR  and  the  Secretary 
of  the  Army  presented  the  Vice  President's  Hammer  Award  to  the  Person¬ 
nel  Demo  team  in  a  ceremony  at  the  Pentagon. 
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7.  A  GPRA  Pilot  Project  in  Performance  Measurement 

In  order  to  assure  themselves  and  the  American  taxpayer  that  the  govern¬ 
ment  was  indeed  delivering  results  for  the  money  it  spent.  Congress 
enacted  a  law  (the  Government  Performance  and  Results  Act  of  1993,  PL 
103-62— GPRA)  requiring  all  agencies  to  operate  on  a  results-oriented 
basis.  That  is,  each  agency  was  to  produce  long-range  strategic  and  annual 
performance  plans  with  quantitative  descriptions  of  outcomes  that  could 
be  expected  as  a  result  of  its  operations.  This  concept,  if  not  easily  accom¬ 
modated  in  many  areas  of  government  operations,  was  particularly  vexing 
for  the  research  community.  However,  seeing  GPRA  as  an  opportunity  to 
instill  businesslike  practices  in  our  R&D  operations,  ARL  volunteered  to 
become  a  pilot  project  under  the  law,  and  to  develop  experimental  tech¬ 
niques  in  managing  a  field  that  has  been  notoriously  resistant  to  manage¬ 
ment,  despite  many  decades  of  work  in  the  area.  The  results  of  our  efforts 
have  been  widely  lauded  as  innovative  advances  in  the  state  of  the  art  in 
R&D  management. 


7.1  Historical  Context  of  ARL's  Participation  as  a  GPRA 
Pilot  Project 

When  ARL  was  activated  at  the  beginning  of  FY93,  the  Acting  Director 
decided  that  beginning  a  new  organization  afforded  an  opportunity  to 
introduce  new  management  practices,  so  that  we  might  function  more  like 
a  private  sector  R&D  laboratory.  These  new  practices  specifically  included 
business  planning  and  performance  evaluation— which,  coincidentally, 
were  the  two  basic  components  of  GPRA. 

Management  planning  and  measuring  has  long  been  studied  for  applica¬ 
tion  to  the  world  of  R&D.  Management  literature  is  replete  with  writings 
going  back  many  decades  in  both  planning  and  measuring.  However,  no 
approaches,  particularly  in  the  area  of  performance  evaluation,  have  ever 
been  found  to  be  completely  satisfactory  for  R&D,  especially  when  com¬ 
pared  to  the  planning  and  measuring  techniques  used  in  other  types  of 
business  functions.  This  has  been  found  to  be  so  not  only  in  government, 
but  in  the  private  sector  as  well  (despite  the  common  view  that  the  "bot¬ 
tom  line"  is  a  suitable  metric  for  all  of  a  company's  activities).  Industry  is 
struggling  with  the  same  issues  that  GPRA  lays  before  the  in-house  gov¬ 
ernment  research  organizations. 

The  reasons  for  this  difficulty  with  R&D  management  are  well  known: 

•  the  likely  outcomes  of  research  are  not  usually  calculable  in  advance; 

•  the  knowledge  gained  from  research  is  not  always  of  immediate  value 
(there  is  a  long  time  lag  between  inputs/outputs  and  outcomes),  nor  can 
that  value  always  be  determined  or  quantified  in  advance; 

•  results  are  more  often  serendipitous  than  predictable; 

•  there  is  a  high  percentage  of  negative  findings  in  research,  which,  while 
often  very  useful  themselves,  are  not  valued  as  outcomes  in  the  GPRA 
sense; 
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•  the  path  by  which  the  results  of  research  are  transitioned  into  products  is 
not  straightforward  and  is  usually  populated  by  a  large  number  of  differ¬ 
ent  individuals  and  organizations;  and,  most  simply, 

•  the  unknown  cannot  be  measured. 

Even  though  the  "D"  of  R&D  is  somewhat  more  amenable  to  planning  and 
measuring  than  the  “R,"  the  difficulty  of  evaluating  R&D  is  still  daunting 
when  compared  to  many  production  or  service  functions. 

Despite  this  background,  and  with  many  research  managers  carrying  the 
scars  of  previous  attempts  at  both  planning  and  measuring  in  the  research 
environment,  ARL  decided  to  try  again  to  produce  a  business  management 
system  that  could  be  tailored  to  the  unique  R&D  environment  and  yet  be 
logical,  rational,  meaningful,  and  above  all,  doable,  and  that  would  add 
value  to  the  laboratory.  When  GPRA  came  along,  ARL  had  invested  almost 
two  years  of  effort  in  the  planning  and  measuring  processes;  at  that  time, 
we  decided  that  we  were  in  as  good  a  position  as  anyone  to  help  the  fed¬ 
eral  R&D  community  determine  how  it  should  be  assessed  by  OMB  and 
Congress.  Therefore,  in  March  1994,  ARL  submitted  a  proposal  to  OMB  to 
become  a  Phase  I  (Performance  Measurement)  pilot  project.  OMB  officially 
designated  the  laboratory  a  Pilot  Project  for  Performance  Measurement  on 
6  July  1994.  This  designation  made  ARL  the  only  R&D  laboratory  in  the 
government  to  be  included  in  the  GPRA  pilot  process.  As  such,  we  have 
had  a  responsibility  to  our  research  colleagues  throughout  the  government 
for  representing  their  interests  as  OMB  and  Congress  worked  towards  the 
government-wide  implementation  of  the  Act. 

7.2  Business  Planning  at  ARL 

A  major  initiative  instituted  at  the  time  of  ARL's  activation  was  a  fairly 
sophisticated  business  planning  process.  This  was  centered  around  a  four- 
volume  business  plan,  the  production  of  which  was  tied  to  the  annual  bud¬ 
get  cycle  of  the  Defense  Department  (fig.  6): 

•  Volume  I,  the  Strategic  Plan ,  looks  out  10  to  15  years  and,  beginning  with 
our  mission  and  vision  statements,  assesses  ARL's  long-term  goals,  the 
business  environment,  the  users'  long-term  requirements,  ARL's  strengths 
and  weaknesses,  etc. 

•  Volume  II,  the  Long-Range  Plan ,  looks  at  the  resourcing  of  the  Strategic  Plan 
over  the  next  six  years  (the  "POM,"  or  Program  Objective  Memorandum, 
period)  in  terms  of  funds,  personnel,  and  facilities,  based  on  projections  of 
the  DoD  budget  process. 

•  Volume  III,  the  Annual  Performance  Plan,  takes  the  strategic  goals  from 
Volume  I  and  the  resources  from  Volume  II,  lays  out  the  projects  and 
resources  for  the  coming  year,  and,  to  the  extent  possible,  sets  quantifiable 
metrics  against  which  performance  will  be  measured. 

•  Volume  IV,  a  corporate-style  Annual  Revieiv,  reports  at  the  end  of  the  year 
the  degree  to  which  the  proposed  goals  in  the  performance  plan  were 
accomplished. 
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Figure  6.  ARL  business 
planning  cycle  coupled 
to  Army  PPBES 
(Planning, 
Programming, 
Budgeting,  and 
Execution  System) 
cycle. 


When  GPRA  was  enacted,  it  became  clear  that  Volumes  I,  III,  and  IV  were 
in  alignment  with  the  Act's  requirements,  with  Volume  II  filling  the  role  of 
a  supporting  document  to  Volume  I.  These  plans  are  developed  and 
updated  at  an  annual  cycle  of  quarterly  meetings  held  by  the  ARL  Director 
and  his  senior  management  team.  The  timing  of  these  meetings  and  the 
resulting  updated  documents  is  tied  to  the  annual  budget  cycle,  so  that  the 
information  is  available  for  submission  through  Army  headquarters  to 
DoD  in  a  timely  fashion. 


7.2.1  ARL  Strategic  Plan 

The  Strategic  Plan  contains  an  introductory  discussion  about  the  ARL  mis¬ 
sion  and  vision,  along  with  some  broad  statements  about  long-range  poli¬ 
cies.  It  continues  with  a  brief  explanation  of  the  plan's  relationship  to  other 
DoD  and  Army  planning  documents  and  processes.  The  last  part  of  the 
introductory  material  is  a  discussion  of  Army  requirements  and  needs  in 
the  near,  mid,  and  far  terms. 

The  heart  of  the  document  is  made  up  of  sections  for  each  of  ARL's  pri¬ 
mary  mission  areas  (digitization  and  communications  science,  armor  and 
armaments,  soldier  system  technology,  vehicle  technology,  and  survivabil¬ 
ity  and  lethality  analysis).  For  each  of  these,  a  statement  of  strategic  intent 

is  followed  by  several  long-term  goals,  each  associated  with  a  desired  out¬ 
come. 

These  missions  are  also  cast  in  terms  of  "Grand  Challenges"  to  the  direc¬ 
torates  which  are  a  way  to  encapsulate  overarching  needs  of  the  Army  into 
strategic  technical  vectors  for  ARL.  The  five  Grand  Challenges  enunciated 
by  our  Director  for  ARL  are 

•  provide  weapons  systems  technology  for  the  Future  Combat  System  (FCS), 

•  provide  lighter,  faster,  more  fuel-efficient  mobile  platforms  to  enhance 
deployability  and  reduce  the  logistics  tail. 
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•  provide  commanders  unprecedented  real-time  situation  awareness  of  the 
battlefield, 

•  significantly  improve  the  battlefield  soldier's  ability  to  absorb  information 
and  make  decisions,  and 

•  solve  the  defensive  information  warfare  problem. 

As  an  example  of  the  association  of  the  strategic  intent  statement  with 
goals  and  outcomes,  take  the  armor /armaments  mission  area.  The  strate¬ 
gic  intent  states:  "  ARL  will  advance  armor  and  armaments  technologies  to 
provide  the  21st  century  soldier  with  the  ability  to  overwhelm  forces  with 
minimum  casualties  and  collateral  damage  and  to  survive  against  a  range 
of  current  and  future  threats."  One  goal  that  follows  from  this  is  to  "create 
designs  for  advanced  armors  ...  that  offer  enhanced  combat  survivability 
and  significantly  reduced  weight,"  with  the  outcome  of  "Increased  protec¬ 
tion  through  lower  cost/lower  weight  passive  armor  concepts." 

This  example  illustrates  the  concept  of  the  "planning  thread,"  which  starts 
from  the  organization's  mission  and  vision  statements  and  can  be  followed 
through  the  planning  process  to  the  individual  S&E  at  the  bench.  In  the 
example  just  cited,  the  goal  of  a  lighter  armor  arises  from  the  fact  that  the 
Army  of  the  future  will  be  primarily  based  in  the  United  States  rather  than 
overseas  and  thus  will  need  to  be  more  readily  deployable  in  time  of  crisis. 
The  current  main  battle  tank  is  too  heavy  for  more  than  one  at  a  time  to  be 
carried  on  an  air  transport.  A  significant  reduction  in  the  weight  of  the 
armor  would  allow  two  to  be  carried  per  plane,  thus  halving  the  number 
of  transports  and  the  amount  of  time  required  to  deploy  an  armored  unit 
overseas,  as  well  as  the  amount  of  fuel  required.  Based  on  this  strategic 
goal,  we  have  a  program  to  develop  a  composite  armor  material  that  will 
provide  ballistic  protection  equal  to  current  armors,  but  weigh  25  percent 
less.  From  such  a  strategic  goal,  we  can  then  derive  a  host  of  short-term 
technical  goals  that  can  be  expressed  in  the  annual  performance  plans  for 
the  next  several  years.  Thus,  the  materials  scientist  working  on  the  chemis¬ 
try  of  composites  can  "follow  the  planning  thread"  to  see  where  his  work 
connects  to  the  overall  mission  of  ARL  and  to  the  Army. 

This  approach  is  obviously  more  amenable  to  requirements-directed  work, 
that  is,  applied  research  or  development.  However,  it  can  also  be  applied, 
at  least  in  a  limited  fashion,  to  our  basic  research  program.  The  assumption 
is  that  all  research,  no  matter  how  basic,  is  undertaken  for  some  purpose 
and  with  some  goal  in  mind,  even  if  far  removed  from  an  end-item 
application. 

The  strategic  plan  is  discussed  and  developed  or  modified  every  year  in 
late  January  at  the  second  quarterly  (or  "Q2")  meeting  of  the  Director  and 
his  senior  staff  and  directorate  heads  (SES  level).  The  timing  allows  for 
major  changes  in  the  organization's  goals  to  be  submitted  to  the  following 
year's  budget  preparation.  The  effort  is  very  much  top  management 
driven,  with  all  the  senior  managers  bringing  to  the  table  their  experiences 
and  contacts  from  the  past  year  with  headquarters,  users,  and  customer 
personnel.  The  Director's  staff  then  transforms  the  results  of  this  meeting 
into  the  actual  document. 
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7.2.2  ARL  Performance  Plan  and  Performance  Report 

The  performance  plan  is  structured  in  three  sections.  The  first  section  deals 
with  management  initiatives  and  goals.  About  a  dozen  goals  are  described 
for  the  coming  fiscal  year,  dealing  with  things  like  the  implementation  of 
the  FedLab  concept  of  operations  for  ARL,  progress  on  major  BRAC  con¬ 
struction  projects,  and  other  significant  management  undertakings.  The 
second  section  covers  the  technical  objectives.  Here  the  heads  of  each  of 
the  technical  directorates  present  their  top  five  objectives  to  be  achieved  in 
the  upcoming  year's  technical  program.  These  objectives  are  only  one  or 
two  sentences  long  and,  to  the  greatest  extent  possible,  state  some  quanti¬ 
fiable  attribute  or  parameter  to  be  attained  (e.g.,  "use  nanofabrication  tech¬ 
niques  to  fabricate  a  2x2  prototype  spatial  light  modulator  for  optical 
image  and  signal  processing.  The  modulator  will  exhibit  picosecond 
switching  speed  with  an  average  contrast  ratio  of  100  when  switched  with 
only  10  V").  Again,  note  that  this  type  of  quantitative  statement  is  certainly 
applicable  to  applied  work,  but  only  in  a  more  limited  sense  to  basic 
research  tasks.  The  third  section  of  the  plan  lays  out  the  year's  goals  for  the 
metrics  that  the  Director  wants  to  emphasize  (metrics  are  discussed  in 
sect.  7.3.2). 

An  ARL-wide  summary  version  of  the  performance  plan  has  been  submit¬ 
ted  to  OMB  every  year  in  compliance  with  the  GPRA  pilot  process.  We  also 
prepare  a  second  version  containing  an  additional  breakout  of  objectives 
and  goals  by  directorate.  Because  of  its  greater  size  and  emphasis  on  detail, 
this  version  is  for  internal  use  only. 

The  annual  performance  report  is  also  structured  in  three  parts.  The  first 
part  is  a  brief  discussion  of  the  overall  state  of  the  lab  and  then  a  fairly 
extensive  review  of  the  past  year's  accomplishments,  technical  and  other¬ 
wise.  The  annual  report  correlates  these  accomplishments  with  each  of  the 
objectives  set  forth  in  the  performance  plan  published  at  the  beginning  of 
the  fiscal  year.  This  presentation  includes  the  quantitative  goals  that  were 
met  or  not  met.  For  example,  in  our  FY93  performance  plan  (pre-GPRA), 
one  goal  stated  "Demonstrate  a  fully  automatic  synthesis  tool  for  creating 
VLSI  [very-large-scale  integration]  chips."  The  FY93  Annual  Review,  pub¬ 
lished  14  months  later,  reported  this  accomplishment:  "Expanded  the  insti¬ 
tutionalization  process  for  electronic  design  automation  methodology 
through  the  development  of  a  computer-aided  design  synthesis  tool  for  VLSI 
circuits." 

The  second  part  of  the  annual  report  contains  a  review  and  analysis  of  the 
lab  s  performance  over  the  past  year.  Fiscal,  personnel,  and  facilities  data 
are  presented  graphically,  with  explanatory  notes  and  discussion.  This  sec¬ 
tion  presents  the  metrics,  again  in  terms  of  the  goals  from  the  performance 
plan  and  the  accomplishments  for  the  year,  along  with  appropriate  discus¬ 
sion  of  the  results  and  any  variances  from  the  plan.  The  final  section  of  the 
report  (published  as  a  separate  volume)  is  a  listing  of  all  the  patents 
received  and  all  the  papers  and  reports  published  during  the  year. 
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Volume  II  of  the  business  plan,  the  Long-Range  Plan,  is  not  submitted  as 
part  of  the  GPRA  pilot  process.  It  is  a  fairly  weighty  document  for  internal 
use  only.  It  describes  in  great  detail  the  fiscal  and  personnel  resourcing  of 
the  strategic  plan.  It  analyzes  the  finances  of  each  directorate  at  the  project 
level  for  the  next  six  years  and  identifies  potential  problem  areas  requiring 
senior  management's  attention. 

7.3  Performance  Evaluation  at  ARL 

A  second  management  innovation  introduced  just  before  ARL's  activation 
was  a  methodology  to  evaluate  the  health  and  performance  of  the  labora¬ 
tory.  R&D  performance  evaluation  has  been  a  problem  that  has  eluded  a 
satisfactory  solution  for  many  decades.  Efforts  in  this  area  have  fallen  into 
one  of  three  approaches: 

•  Retrospective  anecdotal  reviews.  These  are  usually  impressive  and  mean¬ 
ingful  to  the  technical  community,  but  not  to  the  senior  agency  leadership, 
who  want  to  be  able  to  define  specific  progress  towards  goals  in  real  time. 

•  Peer  reviews.  Again  these  are  common  practice  in  the  technical  commu¬ 
nity;  they  are  somewhat  more  acceptable  to  senior  managers,  although  still 
not  completely  satisfying. 

•  Metrics.  Metrics  are  usually  things  that  are  easily  countable,  like  patents 
and  papers;  these  are  usually  not  at  all  satisfactory  in  evaluating  the  true 
worth  of  a  technical  program  or  organization.  To  make  matters  worse, 
from  time  to  time,  complex  schemes  have  been  proposed  that  take  such 
"measures"  and  perform  arcane  algorithmic  operations  on  them  that  often 
obscure  whatever  little  meaning  they  may  have  originally  had. 

In  an  attempt  to  cope  with  this  long-standing  problem,  ARL  developed  its 
Performance  Evaluation  Construct,  which  we  believe  is  a  rational  and  logi¬ 
cal  approach  to  answering  the  questions  of  how  well  the  lab  is  performing 
and  the  degree  of  its  health.  It  is  a  semi-quantitative  approach  that  requires 
the  Director  to  take  a  broad  view  of  many  different  factors,  some  numeric 
and  some  descriptive,  and  then  to  personally  integrate  them  into  a  picture 
that  he  can  present  to  a  variety  of  audiences  (including  stakeholders)  in 
whatever  format  is  appropriate  at  the  moment.  There  are  three  values  or 
principal  areas  of  interest  to  which  the  Construct  responds: 

•  Relevance.  Does  the  work  being  performed  respond  to  some  bona  fide 
requirement  of  a  customer?  That  is,  does  anyone  care  about  the  work  we 
are  doing? 

•  Productivity.  On  any  given  project,  or  for  the  lab  as  a  whole,  is  progress 
being  made  towards  some  specified  goal  at  an  acceptable  rate?  That  is,  are 
we  giving  the  customer  some  product  of  use  to  him  in  a  timely  fashion? 

•  Quality.  Is  the  work  being  done  at  a  level  that  could  be  considered  at  or 
beyond  the  state  of  the  art?  That  is,  is  ARL  a  world-class  institution  doing 
world-class  work? 
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Since  GPRA  requires  that  results  be  described  in  realistic  and  readily 
understandable  terms,  the  Construct  is  clearly  tied  to  the  evaluation 
requirements  of  the  Act.  The  Construct  is  built  on  three  pillars:  peer 
review,  metrics,  and  customer  feedback.  As  indicated  in  figure  7,  these 
techniques  respond  to  the  three  values  above  in  varying  ways. 


Figure  7.  Relationship 
of  three  pillars  of  ARL 

Performance  Method 

Evaluation  Construct 
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7.3.1  Peer  Review 

Contemporary  or  retrospective  peer  review  is  a  generally  accepted 
approach  to  performance  evaluation  throughout  the  world  of  science  and 
technology.  However,  it  has  certain  limitations,  such  as  the  breadth,  depth, 
and  independence  of  the  reviewers.  It  specifically  addresses  quality  and,  to 
a  more  limited  extent,  productivity;  it  is  not  expected  to  respond  to  the 
question  of  relevance  with  any  specificity.  (Prospective  peer  reviews,  often 
used  as  tools  in  grant  selection,  can  respond  to  relevance  if  the  reviewing 
body  so  desires  and  is  appropriately  constituted.)  A  panel  of  recognized 
experts  in  a  scientific  field  is  well-equipped  to  make  judgments  on  the 
technical  quality  of  the  programs  and  the  technical  staff,  and  on  the 
relationship  of  the  facilities  and  equipment  to  the  state  of  the  art.  These 
kinds  of  judgments  are  most  important  for  the  leadership  of  a  laboratory  if 
the  programs  are  to  be  properly  executed  and  the  laboratory  is  to  continu¬ 
ally  improve. 

Taking  a  cue  from  NIST,  ARL  contracted  with  the  National  Research  Coun¬ 
cil  (NRC)  of  the  National  Academies  of  Science  and  Engineering  to  as¬ 
semble  an  ARL  Technical  Assessment  Board  (TAB).  The  TAB  consists  of  14 
individuals  of  international  reputation.  Under  this  board  are  six  panels  of  8 
to  10  people  each,  also  of  high  repute  within  the  technical  community. 
These  panels  review  each  of  our  primary  mission  areas.  The  Board,  with  its 
panels,  provides  an  appraisal  of  the  scientific  and  technical  efforts  of  ARL. 
It  is  specifically  enjoined  from  making  judgments  on  the  programmatic 
structure  of  our  work,  partly  because  it  is  not  equipped  to  perform  this 
function,  and  partly  because  there  are  numerous  other  channels  through 
which  ARL  receives  such  guidance. 

Because  of  the  size  and  diversity  of  the  ARL  technical  program,  the  panels 
review  the  total  program  in  depth  over  a  three-year  period,  reporting 
annually  to  the  TAB  on  the  one  third  of  the  program  reviewed.  The  panels 
produce  a  descriptive  assessment  in  the  form  of  a  written  report  published 
by  the  NRC.  The  TAB  also  meets  annually  with  the  Director  to  provide  an 
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informal  report  of  its  findings  before  the  publication  of  the  report,  and  to 
receive  guidance  on  the  Director's  desires  for  special  areas  of  emphasis  in 
the  next  year's  review.  This  meeting  is  held  just  before  Q2,  so  that  the 
results  of  the  TAB's  review  can  be  used  as  input  to  the  strategic  planning 
process. 

The  initial  contract  with  the  NRC  was  for  three  years  at  approximately 
$650K  per  year.  From  these  funds,  the  NRC  pays  the  expenses  of  all  the 
members  of  the  Board  and  the  panels,  supplies  a  full-time  Executive  Direc¬ 
tor  and  appropriate  clerical  and  support  staff,  and  provides  for  the  logis¬ 
tics  and  publication  requirements  of  the  Board.  This  approach  has  the  ad¬ 
vantages  of  providing  the  independence  and  stature  required  of  a  peer  re¬ 
view,  removing  any  possible  conflicts  of  interest  from  ARL,  and  stamping 
the  endeavor  with  the  NRC's  imprimatur.  The  NRC  has  published  its  first 
and  second  years'  findings  from  the  National  Academy  Press  as  the  1996 
and  1997  Assessments  of  the  Army  Research  Laboratory.  Its  1998  review  is  cur¬ 
rently  in  process. 

7.3.2  Metrics 

The  things  that  are  "countable"  in  the  world  of  R&D,  for  the  most  part,  are 
only  peripherally  connected  to  the  three  values  of  relevance,  productivity, 
and  quality.  Nevertheless,  because  of  the  additional  information  they  can 
provide  (specifically  as  indicators  of  the  functional  health  of  the  organiza¬ 
tion),  metrics  are  judged  to  be  necessary  in  any  evaluation  process.  How¬ 
ever,  they  must  not  be  taken  out  of  context  nor  given  more  importance 
than  they  warrant.  For  this  reason,  ARL  has  eschewed  the  use  of  any  sort 
of  weighting  schemes  or  algorithmic  manipulations  of  these  data.  The 
Director  is  well  qualified  to  personally  integrate  the  values  of  the  different 
metrics,  and  then  to  report  them  in  whatever  format  is  appropriate  for  the 
audience.  Overall  aggregation  into  some  sort  of  "score"  for  the  lab  is  not 
considerd  to  be  productive. 

Accepting  these  limitations,  ARL  assembled  a  set  of  about  60  metrics  for 
consideration.  As  might  be  expected,  most  of  these  were  input  metrics, 
with  several  output  metrics;  metrics  for  outcomes  were,  not  unexpectedly, 
absent.  These  metrics  can  be  aligned  with  the  four  elements  of  the  ARL 
vision  statement  (see  fig.  8).  While  none  of  these  vision  element  groupings 
relate  directly  to  the  requirements  of  GPRA  in  terms  of  measuring  out¬ 
comes,  they  do  provide  a  useful  way  to  look  at  information  about  the 
health  of  the  lab.  Furthermore,  viewing  these  numbers  in  context  can  pro¬ 
vide  critical  insights.  For  example,  two  of  the  metrics  in  the  personnel  area 
are  number  of  staff  members  holding  doctorates  and  the  age  distribution 
of  the  staff.  While  it  is  obvious  that  the  first  of  these  would  be  of  concern  to 
a  laboratory,  age  data  might  seem  of  more  questionable  value.  Yet,  when 
one  ARL  directorate  plotted  doctorates  against  the  age  of  its  workforce,  it 
realized  that  it  had  no  PhD's  on  the  staff  that  were  under  40  years  old.  This 
caused  the  head  of  that  directorate  to  re-think  his  hiring  and  staff  develop¬ 
ment  strategies. 
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Preeminent  in  key  areas  of  science  ... 

Deliverables 

1/  Top  tasks  (%  met) 

✓  Science  and  Technology  Objectives  (STOs) 

(%  met) 

Documentation  (leaving  tracks) 

✓  No.  of  refereed  papers /proceedings 
1 y  No.  of  ARL  technical  reports: 

No.  of  chapters/books  written 
Patents: 

Total  No. 

No.  of  invention  disclosures 

Facilities/equipment 

$  value  capital  equipment  purchased  in  FY 
$M  invested  in  facilities  in  FY 
Replacement  rate  of  facilities 


Staff  widely  recognized  as  outstanding  ... 

Profile 

✓  %  PhDs  (S&Es) 

No.  of  technicians  per  S&E 

Training 

%  employees  with  40+  hr  training 

✓  No.  of  employees  on  long-term  training 

✓  No.  of  employees  on  academic  training 

✓  Total  semester  credit-hours  completed 
PhD  candidates 

Esteem  factors 

No.  of  significant  awards 
No.  of  invited  presentations 
No.  of  prestigious  posts 


Miscellaneous 

Financial 

Obligation 

Disbursement 

IH  (In-House)/ OGA  (other  Government 
Agency) /contract  $ 

Indirect  overhead  ($M) 

G&A  (%  total  revenue) 

Personnel  statistics 

Glidepath 

Average  age  (S&Es;  total) 

Average  grade  (S&Es;  total) 

Average  sick  leave  use  (S&Es;  total) 
Turnover  rate  (S&Es;  total) 

Procurement 

Average  small  purchase  cycle  time 
%  of  Higher  Education  Institutions  (HBCU  / 
MI)  contract  $ 

Administrative/Procurement  Lead  Times 
(ALT/PALT) 


Seen  by  Army  users  as  essential  to  their 
mission  ... 

Technology  transitions 

No.  of  significant  technology  transitions 
Ratings  from  customer  surveys 

✓  Technical  Planning  Annexes  (TPAs) 

✓  Reimbursable  customers 
Users 

Senior  leadership 

Financial 

✓  Reimbursable  customer  orders  ($M) 
Greening  the  workforce 

No.  of  officers 
No.  of  enlisted 

%  employees  completing  "Greening"  course 
No.  employees  completing  EAST  (Field 
Assistance  in  Science  and  Technology)  Jr. 
training 

No.  of  FAST  advisers 


Intellectual  crossroads  for  the  technical 
community ... 

Guest  researchers  out  of  ARL 

Total  No. 

%/  Total  work-year  equivalents 
Average  length  of  stay 
No.  staying  3+  months 

Advisers 

No.  of  NRC  approved  advisers 
Guest  researchers  into  ARL 
\/  Total  No.: 

1 /  No.  of  post-docs 
No.  from  HBCU/MI 
✓  Total  work-year  equivalents 
Average  length  of  stay 
No.  staying  3+  months 
Cooperative  R&D 
No.  of  new  CRADAs 

No.  of  new  Patent  License  Agreements  (PLAs) 
Income  from  CRADAs /PLAs 
No.  of  Technology  Project  Officers  (TPOs) / 
Assistant  TPOs  (ATPOs)  (international) 


Vision  element  categories 

✓  Metrics  included  in  Directors' 
performance  standards 


Figure  8.  FY98  ARL  performance  metrics  grouped 
by  vision  elements. 
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In  general,  60  metrics  are  far,  too  many  to  deal  with  regularly.  The  data  can 
be  collected  and  used  in  various  formats,  but  to  be  useful  a  more  compact 
set  must  be  defined  and  then  brought  to  senior  management's  attention. 
The  Director  accordingly  decided  on  17  that  were  of  special  importance  to 
him  and  that  he  wished  to  use  as  "levers"  to  move  the  organization  in  cer¬ 
tain  directions  and  to  help  set  an  appropriate  environment  in  which  high- 
quality  research  could  be  performed.  These  17  were  singled  out  on  the  ba¬ 
sis  of  the  Director's  judgment  of  what  a  world-class  laboratory  should  look 
like,  a  vision  developed  over  a  long  career  of  managing  R&D  organizations 
(most  recently,  NIST);  these  particular  metrics  were  part  of  that  vision.  For 
instance,  the  Director  believes  that  a  world-class  research  institution 
should  have  about  40  percent  of  its  technical  staff  at  the  doctorate  level. 
When  he  came  to  ARL,  he  found  that  this  number  was  at  22  percent.  Thus, 
he  included  this  particular  metric  among  the  17  for  special  attention. 

He  then  brought  senior  management's  attention  to  the  metrics  by  includ¬ 
ing  them  in  their  performance  appraisals.  ARL's  senior  management  com¬ 
prises  the  (SES-level)  heads  of  the  seven  technical  directorates  and  centers 
who  report  directly  to  the  ARL  Director.  The  Director  negotiates  goals  for 
these  17  metrics  with  the  directorate  and  center  heads  individually,  taking 
into  account  the  nature  of  the  work  in  the  specific  directorate.  For  instance, 
a  directorate  with  a  mission  oriented  more  towards  basic  research  would 
be  expected  to  have  a  higher  goal  for  refereed  journal  articles  and  a  lower 
goal  for  patents,  while  a  more  engineering-oriented  directorate  would 
have  the  opposite  emphasis.  Our  Survivability/Lethality  Analysis  Direc¬ 
torate  (SLAD),  which  concentrates  on  systems  testing  for  Program  Manag¬ 
ers,  would  be  expected  to  have  neither  papers  nor  patents  in  any  substan¬ 
tial  quantity.  Its  metrics  would  include  test  reports  delivered  to  customers. 

In  general,  the  Director  sets  the  goals  for  the  various  metrics  partly  based 
on  his  personal  experience  and  partly  based  on  our  benchmarkings  of 
other  world-class  research  organizations  (see  sect.  8). 

Although  the  career  of  an  SES-level  manager  will  not  rise  or  fall  solely  on 
whether  his  directorate  makes  its  quota  of  patents  or  papers,  placing  these 
metrics  in  his  performance  standards  does  draw  his  attention  to  them  and 
has  had  a  noticeable  effect  on  the  overall  performance  of  ARL  in  these 
selected  areas — as  was  the  Director's  intention. 

At  the  end  of  the  year,  the  statistics  are  gathered  and  reported  to  the  Direc¬ 
tor,  who  takes  them  into  account  as  he  prepares  the  performance  apprais¬ 
als  of  his  senior  managers.  These  figures  are  also  rolled  into  an  overall  ARL 
picture.  The  effect  of  this  process  is  demonstrated  by  the  example  of  the 
percentage  of  doctorates  on  the  technical  staff,  which  has  risen  from  22  to 
27  percent  in  three  years.  In  light  of  the  various  downsizing  efforts  (hiring 
restrictions,  early  retirements,  buyouts)  that  the  lab  is  undergoing,  this  is 
real  progress  towards  the  long-term  goal  of  40  percent.  Other  effects  of  in¬ 
cluding  metrics  from  the  Director's  "short  list"  in  the  senior  managers' 
performance  appraisals  are  the  following  percentage  changes  since  1993: 

•  number  of  guest  researchers  into  ARL — up  48  percent 

•  number  of  guest  researchers  out — up  293  percent 

•  number  of  NRC  post-doctoral  fellows — up  94  percent 
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•  number  of  refereed  journal  articles — up  155  percent 

•  number  of  technical  reports — up  10  percent 

•  number  of  employees  on  long-term  training— up  127  percent 

Obviously  the  long-term  outcomes  of  ARL's  programs  will  stem  from  its 
technical  accomplishments.  While  these  are  usually  considered  in  terms  of 
shorter  term  intermediate  outputs,  they  are  among  the  more  important  of 
the  metrics  that  we  can  collect.  These  include  a  variety  of  "countables"  that 
are  defined  differently  for  the  different  directorates.  However,  they  all  boil 
down  to  counting  how  many  of  the  specific  technical  goals  set  at  the  begin¬ 
ning  of  the  fiscal  year  were  actually  accomplished.  Keep  in  mind  that  the 
quality  and  value  of  these  individual  goals  are  not  evaluated  by  the 
metrics,  but  rather  by  the  other  two  pillars  of  the  Construct — peer  review 
and  customer  feedback.  However,  the  metrics  provide  a  handy  scorecard 
to  keep  track  of  how  well  the  head  of  a  directorate  is  doing  in  fulfilling  his 
annual  obligations. 

Many  of  the  other  metrics  are  directed  towards  ascertaining  the  technical 
atmosphere  and  operational  health  of  the  laboratory.  An  example  is  a 
group  of  metrics  called  "esteem  factors."  The  assumption  is  that  in  a 
world-class  laboratory  of  over  2100  people,  more  than  half  of  whom  are 
S&Es,  there  should  be  a  certain  number  of  fellows  of  national  societies, 
adjunct  professors  at  prestigious  universities,  winners  of  major  awards, 
etc.  Appropriate  goals  for  such  factors  are  determined  by  comparison  with 
other  world-class  laboratories.  While  such  statistics  may  not  be  under  the 
direct  control  of  management,  the  lack  of  any  such  honors  among  the  staff 
(an  extreme  example)  would  certainly  be  an  indication  that  the  scientific 
atmosphere  at  the  laboratory  is  somehow  lacking  and  needs  attention. 
Similarly,  the  various  input  metrics  (fiscal  performance,  facilities,  person¬ 
nel  data)  provide  indicators  of  organizational  health  to  the  Director. 

Those  metrics  not  regularly  reported  to  the  Director  are  tracked  as  part  of 
normal  business  procedures  (many  of  them  by  automated  systems)  by  the 
various  functional  managers.  They  are  not  surfaced  to  senior  management 
unless  they  begin  to  fall  outside  some  envelope,  and  it  is  deemed  that 
some  form  of  executive  attention  is  required. 

The  linkage  between  all  these  metrics  and  the  outcomes  of  a  laboratory's 
endeavors  is  certainly  tenuous,  but  no  more  so  than  in  industry.  A  great 
deal  of  work  has  been  published  in  this  area,  much  of  it  summarized  in 
publications  by  the  Industrial  Research  Institute.  This  body  of  work  shows 
that  although  various  metrics  can  be  defined  in  financial  terms,  for  R&D 
the  collection  and  reliability  of  these  metrics,  and  their  relationship  to  the 
company's  profits,  are  extremely  problematic. 

7.3.3  Customer  Feedback 

The  third  pillar  of  the  Construct  incorporates  the  concept  of  a  customer 
into  the  research  effort.  While  this  focus  is  certainly  familiar  in  the  world  of 
development  and  production,  it  is  somewhat  foreign  to  the  world  of 
research.  Nevertheless,  if  someone  is  being  paid  for  some  work,  research  or 
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otherwise,  the  payer  presumably  expects  a  product  from  the  payee.  That 
payer  is  the  customer. 

ARL  has  developed  this  notion  by  the  use  of  a  model  of  the  "stakeholders" 
for  the  research  enterprise,  developed  by  Professor  Edward  B.  Roberts  at 
the  MIT  Sloan  School  of  Management.*  According  to  this  model,  in  indus¬ 
try  there  are  three  groups  of  stakeholders  for  the  research  effort:  two 
within  the  firm  and  one  outside  (fig.  9).  The  most  obvious  group  within  the 
firm  is  the  development  and/or  manufacturing  and/ or  marketing  depart¬ 
ments,  who  depend  on  the  output  of  the  research  group  for  the  raw  mate¬ 
rial  (ideas  and  technologies)  that  will  become  the  basis  for  new  product 
lines.  The  external  stakeholder  is,  obviously,  the  end  user  or  customer  of 
the  firm.  However,  the  other  internal  stakeholder,  which  might  not  be  as 
obvious,  is  the  senior  leadership  of  the  firm,  the  Chief  Executive,  Chief  Fi¬ 
nancial  Officer,  or  Chief  Technical  Officer,  who  needs  the  concepts  under 
investigation  now  in  the  research  arm  of  the  firm  to  enable  the  setting  of 
the  long-term  strategic  vectors  for  the  company. 

Translating  this  model  to  Army  acquisition,  we  have  customers  for  the 
immediate  short-term  products  of  our  efforts:  the  RDECs  and  the  various 
system  Program  Managers  and  Program  Executive  Officers  (PMs/PEOs). 
These  customers  require  specific  products  from  the  research  organizations 
to  fit  into  their  development  programs.  Like  the  private  sector,  ARL  also 
has  an  ultimate  end-item  user — the  individual  soldier  in  the  fighting  units. 
While  we  may  not  deliver  a  research  product  to  this  ultimate  customer,  we 
must  be  ever  mindful  of  his  real-world  needs  and  requirements  as  we  con¬ 
duct  our  research  on  concepts  for  systems  that  might  appear  on  a  battle¬ 
field  10  or  20  years  from  now.  (Part  of  maintaining  this  awareness  of  the 
soldier's  needs  is  gained  through  the  ARL  "Greening"  program,  atten¬ 
dance  at  which  is  one  of  the  metrics  shown  in  fig.  8.)  Finally,  again  in  par¬ 
allel  with  industry,  we  have  a  senior  leadership  that  must  look  farther  into 
the  future  than  a  PM,  for  instance,  who  is  concentrating  on  fielding  a  sys¬ 
tem  in  the  next  year  or  two.  The  Chief  of  Staff  of  the  Army,  for  example, 
must  shape  the  fundamental  concepts  of  our  fighting  forces  for  the  next 
several  decades.  He  becomes  a  stakeholder  in  the  laboratory,  since  the 
most  advanced  technologies  being  worked  on  today  will  shape  not  only 


Figure  9.  Roberts' 
model  of  researcher's 
stakeholders. 


The  external  customer 
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* Roberts ,  Edward  B.,  " Benchmarking  the  Strategic  Management  of  Technology — II:  R&D  Performance  ”  Interna¬ 
tional  Center  for  Research  on  the  Management  of  Technology,  MIT  Sloan  School  of  Management,  working  paper  119- 
95,  January  1995. 
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the  systems  that  appear  on  the  future  battlefield,  but  the  combat  doctrine 
and  force  structure  as  well. 

For  that  segment  of  our  customers  to  whom  we  deliver  specific  items,  we 
use  a  series  of  feedback  questionnaires,  not  unlike  what  industry  uses,  to 
sense  the  degree  of  their  satisfaction.  We  have  completed  four  annual 
cycles  of  questionnaires  to  our  internal  development  customers:  to  the 
RDECs  for  whom  we  do  work  at  no  cost,  and  to  other  agencies  that  come 
to  us  on  a  reimbursable  basis  for  specific  tasks.  These  questionnaires  are 
targeted  at  those  individuals  in  these  other  organizations  who  are  respon¬ 
sible  for  specifying  the  product  to  be  delivered.  After  the  work  is  finished, 
each  customer  is  asked  to  rate  our  performance  on  a  l-to-5  scale  in  terms  of 
the  quality,  timeliness,  utility,  etc,  of  the  deliverable.  There  is  also  a  place 
for  comments.  Our  Director  has  established  a  policy  that  any  rating  of  less 
than  3  on  any  item,  or  any  negative  comment,  must  be  responded  to  per¬ 
sonally  by  the  appropriate  SES  level  directorate  or  center  head  within  one 
week.  Aside  from  this  feedback  process  itself,  the  questionnaire  scores  are 
also  a  part  of  the  metrics  process  and  are  included  as  a  basis  for  the  senior 
leaders'  performance  appraisals.  We  send  out  approximately  400  surveys  a 
year  and  have  a  response  rate  of  almost  60  percent.  Our  overall  score  has 
risen  from  3.9  (out  of  5)  to  4.3  over  the  past  four  years. 

We  do  not  use  questionnaires  on  the  other  two  groups  of  stakeholders _ 

the  end  users  and  the  senior  Army  leadership — since  we  do  not  deliver  a 
specific  identifiable  product  directly  to  either  of  these  groups.  However, 
we  have  established  a  Stakeholders'  Advisory  Board  (SAB)  that  brings  to¬ 
gether,  at  the  three-star  level,  the  Army's  senior  leadership  and  user  repre¬ 
sentatives  into  a  forum  that  enables  ARL  to  receive  first-hand  guidance 
and  feedback  from  these  other  two  important  segments  of  our  stakehold¬ 
ers.  The  SAB  has  held  three  annual  meetings  to  date. 

7.3.4  Applicability  of  Evaluation  Techniques 

The  survey  process  for  customer  feedback  is  more  applicable  to  the 
applied  research  that  we  do— the  requirements-driven  R&D— than  to  the 
more  basic,  opportunity-driven  research.  Peer  review,  on  the  other  hand,  is 
more  applicable  to  our  basic  research  work.  Metrics  are  applicable  with 
varying  emphasis,  depending  on  the  metric  and  when  it  is  applied.  In 
other  words,  the  three  pillars  of  the  Performance  Evaluation  Construct  are 
given  different  weights  at  different  points  in  the  R&D  life  cycle. 

The  three  pillars  of  the  ARL  Performance  Evaluation  Construct  are  related 
in  varying  ways  to  the  areas  of  interest  for  evaluation:  quality,  relevance, 
and  productivity.  Peer  review,  as  realized  by  the  Technical  Assessment 
Board,  provides  feedback  on  the  quality  of  the  technical  program.  The 
Stakeholders  Advisory  Board  provides  feedback  on  the  programmatic 
and  managerial  performance  of  ARL,  which  applies  to  relevance  and  pro¬ 
ductivity.  Our  survey  process  also  inquires  into  relevance  and  productivity 
from  the  standpoint  of  those  customers  who  receive  specific  deliverables 
from  us.  The  surveys  apply  at  the  individual  project  level,  the  TAB  ad¬ 
dresses  the  program  level,  and  the  SAB  takes  a  strategic  viewpoint. 
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Although  there  may  be  quantitative  aspects  to  all  this  information,  neither 
it  nor  our  system  of  metrics  is  reduced  to  a  single  ARL  "score."  Rather,  the 
information  is  digested  and  integrated  by  our  Director  for  use  in  a  variety 
of  ways.  This  approach  is  in  consonance  with  the  basic  principles  of  the 
NPR,  which  aims  to  place  the  responsibility  for  managing  with  the  senior 
managers  and  free  them  from  mindless,  bureaucratic  rote. 

7.4  Issues  and  Lessons  Learned 

A  major  issue  presented  by  the  GPRA  to  many  government  organizations 
is  the  difference  in  scale  between  agencies  and  how  the  requirements  of  the 
Act  will  be  translated  to  the  working-level  organizations  within  agencies. 
The  Act  defines  the  agencies  responsible  for  reporting  these  various  plans 
as  the  cabinet-level  organizations.  Thus,  in  our  case,  the  reporting  agency 
is  the  Department  of  Defense.  Depending  on  how  one  counts  the  levels  in 
the  chain  of  command,  ARL  is  5  to  10  levels  below  the  level  where  the  stra¬ 
tegic  and  performance  plans  will  be  drawn  up  for  Congress.  Thus,  for  all 
practical  purposes,  ARL  will  not  be  very  visible  in  the  aggregation  of  data 
at  the  DoD  level.  Some  of  the  civilian  agencies,  which  are  much  smaller 
than  DoD,  may  well  have  their  R&D  functions  exposed  in  somewhat  more 
detail  at  the  OMB  and  Congressional  level.  It  also  needs  to  be  recognized 
that  the  ways  in  which  the  various  GPRA  processes  are  implemented  at  the 
organizational  level  are  dramatically  different  from  the  way  they  are  at  the 
agency  level. 

Along  the  way  we  have  gathered  other  insights  into  the  GPRA  and  related 
processes: 

1.  A  new  mindset  is  required,  especially  in  the  world  of  research.  Thinking  in 
terms  of  outcomes  rather  than  inputs  or  outputs,  effectiveness  as  well  as 
efficiency,  and  customer  satisfaction,  is  somewhat  alien  to  the  R&D  busi¬ 
ness.  However,  for  GPRA  these  concepts  must  be  understood,  translated 
into  workable  constructs  for  the  R&D  community,  and  embraced.  Training 
may  help,  but  in  some  cases  a  "religious  conversion"  will  be  required. 

2.  Senior  management  must  buy  into  the  process  totally.  This  acceptance  can¬ 
not  be  driven  from  the  bottom  up,  since  it  is  intimately  connected  with  the 
very  heart  of  an  organization's  business.  If  the  senior  leadership  does  not 
embrace  this  philosophy  as  part  of  their  planning  process,  they  will  fail  in 
their  attempts  to  use  planning  and  measuring  techniques  as  a  part  of  the 
management  of  their  organizations. 

3.  Participation  in  this  type  of  planning  and  measuring  requires  a  significant 
investment  in  personnel,  funds,  and  time.  Despite  the  original  protesta¬ 
tions  of  those  who  testified  before  Congress  for  the  establishment  of  the 
Act,  it  is  very  labor  intensive  at  all  levels  of  the  organization.  ARL  dedi¬ 
cates  many  work-years  of  effort  to  this  process,  and  devotes  a  significant 
number  of  full-time  permanent  positions  to  the  planning  and  measuring 
functions.  If  this  effort  were  being  expended  solely  to  satisfy  GPRA,  it 
would  have  met  with  a  great  deal  of  resistance  at  ARL.  However,  since  we 
began  the  planning  and  measuring  efforts  independently  for  reasons  of 
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good  business  practice,  GPRA  has  not  become  a  major  additional  factor  in 
the  cost  equation  for  us.  But  it  should  not  be  forgotten  that,  however  moti¬ 
vated,  this  kind  of  management  does  not  come  cheap. 

One  might  consider  that  political  or  bureaucratic  risks  are  incurred  by  an 
organization  exposing  itself  to  such  in-depth  scrutiny  through  publishing 
this  array  of  planning  and  reporting  documents.  However,  even  if  such 
scrutiny  were  not  required  by  statute,  ARL  believes  that  any  organization 
that  aspires  to  be  world  class  must  be  able  to  withstand,  and  learn  from, 
critiques  by  its  peers  and  stakeholders.  Otherwise,  the  entire  endeavor  is 
pointless. 

As  a  side  note,  except  for  the  NRC's  role  in  our  peer  review  system,  ARL 
chose  not  to  use  contractors  to  support  this  effort,  since  organizational 
planning  is  an  inherently  governmental  function.  We  felt  that  no  contractor 
could  know  as  much  about  our  own  business  as  we  do.  It  is  also  arguable 
whether  we  could  get  the  same  level  of  effort  from  a  contractor  for  much 
less  than  we  are  expending  ourselves. 

4.  A  competent  corporate  information  system  is  essential  to  managing  the 
performance  evaluation  process.  We  have  learned  this  the  hard  way,  since 
we  do  not  have  one  at  this  time,  and  just  doing  the  metrics  work  by  hand  is 
enormously  time-consuming. 

5.  The  application  of  these  techniques  to  basic  research  is  more  difficult  than 
it  is  to  applied  research.  Both  planning  and  evaluating  basic  research  (i.e., 
research  that  is  not  requirements-directed)  is  recognized  as  very  difficult. 
At  ARL  we  have  approached  the  problem  by  assuming  that  all  work  is 
done  for  some  purpose  and  for  somebody.  In  basic  research,  that  purpose 
and  person  may  not  be  immediately  evident.  It  may  be  the  Director  him¬ 
self,  or  it  may  be  "second  order":  an  output  from  some  basic  research  may 
be  an  input  to  some  applied  research  that  does  have  an  identifiable  appli¬ 
cation  and  customer.  Nevertheless,  if  a  purpose  and  a  customer  can  be 
defined,  then  the  rest  of  the  process  follows  more  or  less  directly. 

6.  Metrics  are  a  very  contentious  issue.  In  general,  good  business  practice  dic¬ 
tates  that  a  half  dozen  meaningful  metrics  are  all  that  should  be  used  in  the 
management  of  an  organization.  However,  in  R&D  we  have  concluded 
that  there  simply  are  no  meaningful  outcome  metrics,  in  the  sense  that  a 
manufacturing  or  service  company  would  use  them.  We  did  not  arrive  at 
this  conclusion  capriciously.  On  the  contrary,  we  undertook  extensive  face- 
to-face  interviews  with  the  senior  leadership  of  many  of  our  leading 
technology  companies.  In  every  case,  they  could  proudly  explain  their 
metrics-related  management  processes  in  relation  to  their  product  devel¬ 
opment  and  production  operations;  when  pressed  to  discuss  their  research 
function,  however,  they  all  admitted  that  they  had  no  satisfactory 
approach,  and  that  they  relied  heavily  on  the  intuition  of  their  Chief  Tech¬ 
nical  Officers. 

Whenever  we  discuss  our  60  or  so  metrics,  we  immediately  receive  criti¬ 
cism  for  their  quantity  and  lack  of  relevance.  Therefore,  to  emphasize 
again:  all  these  "countables"  are  used  only  as  indicators  of  the  health  of 
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various  functions  within  the  laboratory.  They  serve  only  limited  purposes, 
but  these  purposes  are  often  important  in  the  management  of  the  R&D  en¬ 
terprise: 

•  Certain  of  the  metrics  serve  as  indicators  in  narrow  functional  areas  (such 
as  fiscal  performance)  and  never  need  to  be  surfaced  to  the  laboratory's 
leadership  unless  they  begin  to  move  outside  some  predetermined 
bounds. 

•  Certain  metrics  may  be  used  by  the  Director  as  "levers"  to  move  the  orga¬ 
nization  in  directions  that  he  feels  are  necessary  (such  as  the  percentage  of 
PhD's  in  a  research  organization). 

•  Finally,  certain  stakeholders  require  data  that,  for  reasons  that  may  not  be 
known  to  us,  have  significance  to  them. 

This  last  purpose  is,  in  a  sense,  the  most  difficult  to  understand,  so  we  just 
accept  it.  Again  and  again,  whenever  we  have  reexamined  the  set  of 
metrics  and  decided  to  drop  one  as  being  of  little  value,  we  will  invariably 
receive  a  question  from  a  stakeholder  concerning  that  particular  number. 
Therefore,  our  philosophy  is,  "If  it's  countable,  we  count  it!"  We  just  take 
care  to  assure  that  such  data  are  not  misused  or  misinterpreted. 

7.  Different  stakeholders  may  have  widely  diverse  views  on  what  is  impor¬ 
tant  for  ARL  to  accomplish.  There  are  internal  and  external  views,  short¬ 
term  and  long-term  views,  emphasis  on  purely  scientific  output  versus 
technical  deliverables  versus  operational  (fiscal,  personnel,  etc)  perfor¬ 
mance,  and  so  on.  The  Performance  Evaluation  Construct  was  designed  to 
provide  the  Director  with  the  required  flexibility  to  respond  across  these 
various  interests. 

Finally,  this  is  still  work  in  progress  for  ARL.  We  made  a  major  advance  in 
the  latest  version  of  our  strategic  plan  by  incorporating  for  the  first  time 
strategic  technical  goals,  in  addition  to  the  more  familiar  broad  manage¬ 
ment  goals  usually  seen  in  such  plans.  In  fact,  this  drastically  revised  plan 
was  motivated  in  part  by  our  receiving  a  new  mission  assignment  from  the 
former  Army  Chief  of  Staff:  to  develop  the  scientific  underpinning  for  the 
future  digitized  battlefield.  That  led  to  a  whole  new  structure  for  our  pro¬ 
gram,  and  with  it,  the  new  FedLab  strategic  approach  to  our  modus  oper- 
andi  (discussed  in  sect.  2). 

Still  needing  improvement  is  the  all-important  linkage  from  strategy  to 
annual  performance.  While  each  of  the  volumes  of  the  business  plan  is 
complete,  consistent,  and  well-structured  within  itself,  the  natural  flow  or 
driving  function  from  one  to  the  other  is  not  always  obvious.  Also,  strate¬ 
gic  human  resource  planning,  facilities  master  planning,  etc,  are  not  inte¬ 
grated  into  the  overall  strategic  plan.  In  addition,  the  actual  application  of 
the  plan  is  not  optimum.  Deployment  down  to  the  workforce  and  first- 
level  supervisors  is  particularly  spotty.  Therefore,  we  will  be  undertaking  a 
senior-level  review  of  the  planning  process  to  see  what  improvements  can 
be  implemented. 


53 


Our  set  of  metrics  must  also  be  continuously  reevaluated  in  terms  of  their 
value  to  the  process,  their  collectability,  their  amenability  to  validation, 
and  most  importantly,  the  message  they  send  to  the  workforce.  It  has  often 
been  observed  that  "what  gets  measured,  gets  done."  We  need  to  assure 
ourselves  that  we  are  measuring  the  right  things  so  that  we  will  do  the 
right  things. 

With  the  submission  of  the  FY96  Annual  Review  to  OMB,  ARL's  participa¬ 
tion  as  a  GPRA  pilot  project  concluded.  Because  of  the  scale  question  dis¬ 
cussed  above,  the  GPRA  per  se  may  not  ultimately  have  a  major  impact  on 
ARL's  business  operation.  However,  the  planning  and  measuring  that  con¬ 
stitute  the  GPRA  process  are  still  important  to  us  and  will  continue, 
regardless  of  the  specifics  of  the  eventual  implementation  of  the  Act.  We 
intend  to  continue  to  use  these  tools  and  improve  upon  them,  simply 
because  it  is  "good  business"  to  do  so. 
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8.  Benchmarking  ARL 

In  the  first  of  two  separate  efforts  related  to  the  management  of  ARL,  we 
undertook  a  benchmarking  study  to  gain  a  better  understanding  of  the 
health  of  our  laboratory.  The  problem  was  that  determining  "health"  on  an 
absolute  basis  was  difficult  to  do  without  having  some  sort  of  guidelines 
or  goals.  Benchmarking  has  enabled  us  to  determine  such  goals  by  observ¬ 
ing  the  characteristics  of  the  "best  in  class"  among  world-class  research 
organizations.  We  were  pleased  to  discover  that  in  many  of  the  areas  exam¬ 
ined,  ARL  compared  favorably  with  its  peers. 

8.1  Early  Efforts  at  Benchmarking 

Benchmarking  is  not  so  much  a  management  initiative  as  it  is  a  process 
that  we  felt  obliged  to  undertake  in  order  to  better  understand  our  own 
business.  Since  there  are  so  few  absolutes  in  running  a  research  organiza¬ 
tion,  peer  comparison  is  one  of  the  few  ways  to  know  if  we  are  heading  in 
the  right  direction.  The  data  from  benchmarking  also  provides  a  means,  in 
addition  to  the  Director's  intuition,  of  setting  goals  for  certain  of  the 
metrics  described  in  the  previous  section. 

We  have  been  undertaking  various  benchmarking  efforts  since  before  ARL 
was  activated.  However,  in  the  sense  that  I  am  using  the  word,  bench¬ 
marking  is  not  the  formal  multistep  process  taught  in  business  schools. 
Rather  it  is  merely  a  simple  comparison  of  various  attributes  of  our  organi¬ 
zation  with  the  same  attributes  of  organizations  widely  recognized  as 
"world  class"  or  "best  in  class,"  the  goal  being  to  see  how  we  stack  up  to 
our  peers,  and  whether  there  are  obvious  soft  spots  that  we  need  to  im¬ 
prove. 

One  of  our  earlier  efforts  occurred  in  1991  when  AMC,  our  parent  com¬ 
mand,  began  a  thrust  to  apply  TQM  principles  to  all  its  functions,  includ¬ 
ing  R&D.  The  predecessor  organization  to  ARL,  LABCOM  (Laboratory 
Command),  was  given  the  responsibility  to  devise  the  appropriate  "tech¬ 
nology  generation  processes"  for  all  of  AMC.  In  the  process  of  doing  that, 
we  contacted  the  following  organizations  to  benchmark  their  approaches 
to  managing  their  corporate  research  functions: 

•  Bell  Labs  (on-site  visit) 

•  Corning  Glass  (on-site  visit) 

•  Motorola 

•  Westinghouse 

•  Martin  Marietta  (on-site  visit) 

•  United  Technologies 

Concerning  the  application  of  TQM  methodologies  to  the  management  of 
the  corporate  laboratories,  we  learned  that  these  industry  leaders 
employed  a  variety  of  techniques  in  their  development,  production,  mar¬ 
keting,  and  support  functions.  However,  when  pressed  specifically  about 
applying  TQM  to  their  research  functions,  every  one,  without  exception, 
said  that  they  did  not  know  how  to  do  that,  and  for  the  most  part,  they 
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relied  on  the  intuition  of  their  technical  managers.  (This  was  very  satisfy¬ 
ing  to  hear,  in  a  perverse  sort  of  way.) 

More  recent  efforts  in  benchmarking  have  been  to  gather  data  from  a  vari¬ 
ety  of  peer  organizations  concerning  management  processes  and  output 
measures.  These  organizations  have  included 

•  NIST, 

•  Naval  Research  Laboratory  (NRL), 

•  Hughes  Research  Laboratory, 

•  RCA  Sarnoff  Labs,  and 

•  Lawrence  Livermore  National  Laboratory. 

We  also  made  several  extended  trips  to  the  United  Kingdom  to  benchmark 
a  wide  variety  of  processes  against  the  Defence  Evaluation  and  Research 
Agency  (DERA),  the  UK's  Ministry  of  Defence  corporate  laboratory. 

Our  functional  support  organizations  have  been  very  active  in  bench¬ 
marking  their  activities  with  the  private  sector.  We  also  have  worked  with 
the  Industrial  Research  Institute  (IRI),  a  trade  association  representing  85 
percent  of  the  commercial  R&D  conducted  in  this  country;  the  Federal 
Research  Assessment  Network  (FRAN),  run  out  of  the  National  Science 
Foundation;  and  an  ad  hoc  R&D  roundtable  made  up  of  representatives  of 
R&D  organizations  from  throughout  the  federal  government. 

Finally,  during  FY95  we  contracted  with  CHI  Research,  Inc.,  to  perform  an 
in-depth  analysis  of  our  patent  output  and  citation  activity  in  comparison 
with  14  government,  industry,  and  university  laboratories.*  These  in¬ 
cluded  NRL,  Los  Alamos,  Livermore,  Oak  Ridge,  NIST,  JPL,  SRI,  Bell  Labs, 
Stanford,  MIT,  Xerox  PARC,  IBM,  and  two  foreign  labs — Julich  and  Toyota. 
The  summary  conclusions  were  as  follows: 

•  ARL  has  the  most  patents  per  active  inventor;  i.e.,  ARL  inventors  are  more 
productive  than  their  counterparts. 

•  However,  because  of  the  more  specialized  nature  of  the  government  work 
(especially  such  ARL  work  as  armor  and  armaments),  ARL's  patents  are 
cited  far  less  than  those  of  its  commercial  counterparts. 

•  In  numbers  of  patents,  ARL  is  comparable  to  NRL  and  ahead  of  the  other 
government  labs,  but  it  is  much  weaker  in  its  indicators  of  citation  and 
linkage  to  nonpatent  references  (journal  articles,  which  indicate  how  close 
to,  or  far  from,  leading-edge  science  its  patents  are). 

•  In  several  specific  categories,  however,  ARL  is  a  leader;  those  categories 
are,  not  surprisingly,  related  to  ordnance  and  armor  materials. 

A  similar  study  was  also  done  by  CHI  on  our  journal  article  portfolio/  this 
time  comparing  us  to  seven  other  government  labs,  five  universities,  and 
five  industrial  labs.  The  conclusions  of  this  study  are  summarized  below: 


* Anthony  Breitzman,  Margaret  Cheney ,  and  John  Perko,  " Assessment  of  ARL  Patent  Portfolio,"  Final  Report  CHI 
Research,  Inc.,  Haddon  Heights,  NJ,  28  December  1 994. 

f  Anthony  Breitzman,  "Assessment  of  ARL  Literature  Portfolio,"  Final  Report,  CHI  Research,  Inc.,  Hadden  Heights 
NJ,  25  September  1995.  5  ' 
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•  ARL's  publishing  activity  has  been  steadily  increasing  over  the  past  five 
years. 

•  ARL  ranks  very  high  in  both  papers  and  patents  per  million  dollars  of  bud¬ 
get  (see  the  discussion  of  the  results  figures  in  the  next  section)  compared 
to  the  other  labs. 

•  Overall,  ARL's  papers  are  not  as  highly  cited  as  would  be  preferred,  but  in 
certain  core  areas  of  research,  ARL's  citations  are  quite  impressive. 


8.2  Recent  Results 

Most  recently,  we  have  just  completed  a  survey  of  the  following  15  organi¬ 
zations  which,  in  one  way  or  another,  represent  first-rate  corporate  or  cen¬ 
tral  research  organizations,  not  unlike  what  ARL  purports  to  be  for  the 
Army: 

•  NRL 

•  Air  Force  Rome  Lab 

•  Air  Force  Wright  Lab 

•  Air  Force  Phillips  Lab 

•  Air  Force  Armstrong  Lab 

•  Lincoln  Laboratories 

•  NIST 

•  NASA  Goddard  Space  Flight  Center 

•  NASA  Langley  Research  Center 

•  Agricultural  Research  Service  Beltsville  Research  Facility 

•  Bell  Labs 

•  SRI,  Sarnoff  Labs 

•  IBM,  Watson  Labs 

•  Lawrence  Livermore  National  Laboratory 

•  Los  Alamos  National  Laboratory 

We  gathered  data  on  their  workforce  population,  their  financial  situation, 
and  their  technical  output.  We  combined  this  with  data  from  the  National 
Science  Foundation/  which  yielded  a  national  perspective,  and  with  the 
1995  annual  IRI  survey  of  its  member  organizations.+  Figures  10  through 
17  display  some  of  the  results. 

Figure  10  shows  that  strictly  by  virtue  of  size  of  workforce,  ARL  compares 
most  directly  with  Phillips  Lab,  Lincoln  Labs,  Wright  Lab,  IBM  Watson 
Lab,  NIST,  NASA  Goddard,  and  NRL.  Sarnoff,  Rome,  and  Armstrong 
clearly  are  smaller,  while  Los  Alamos,  Livermore,  and  the  Agricultural 
Research  Service  labs  are  a  quantum  larger. 

Figure  11  bears  out  our  Director's  view  that  a  world-class  research  organi¬ 
zation  should  have  about  40  percent  of  its  technical  staff  at  the  doctoral 
level.  Of  the  labs  surveyed,  ARL  is  fifth  from  the  lowest  in  percentage  of 
PhD's,  and  is  significantly  below  both  NRL  and  NIST,  the  two  labs  most 
often  compared  to  ARL.  Both  of  these  have  about  50  percent  PhD's. 

*National  Science  Board,  "Science  and  Engineering  Indicators — 1996,"  National  Science  Foundation,  Washington , 
DC. 

* Industrial  Research  Institute,  " IRI/CIMS  Annual  R&D  Survey — FY 92-95,"  Washington,  DC. 
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Figure  10.  Size  of 
laboratory:  total 
personnel. 


Figure  11. 
Percentage  of  S&E 
staff  with  PhD 
degrees. 
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Figure  12  relates  to  the  next  section's  discussion  of  overhead.  This  figure 
shows  the  percentage  of  S&Es  in  the  total  workforce.  This  is  what  the  mili¬ 
tary  might  call  the  tooth  to  tail  ratio.”  Here  ARL  compares  quite  favor¬ 
ably,  having  a  percentage  greater  than  56  percent.  The  NSF  book  tells  us 
that  the  comparable  figure  for  R&D  throughout  the  whole  federal 
workforce  is  only  15  percent,  and  for  the  total  national  scientific  workforce, 
it  is  just  under  30  percent.  (However,  the  annual  IRI  survey  of  its  members 
gives  a  figure  of  74  percent.) 

Figures  13  to  16  compare  the  "output  efficiency”  of  these  organizations,  in 
terms  of  papers  and  patent  awards  per  S&E  and  per  R&D  dollar  (in  mil¬ 
lions).  For  patents,  ARL  again  compares  quite  favorably  (except  for  the 
anomalously  high  number  of  patents  per  S&E  for  the  IBM  Watson  lab; 
when  this  number  was  rechecked  with  IBM,  the  answer  was  "We  put  a  lot 
of  emphasis  on  patenting").  ARL  also  seems  to  be  fairly  efficient  in  produc¬ 
ing  refereed  papers  for  the  money  invested,  although  it  is  not  as  produc¬ 
tive  per  person  as  we  might  like. 
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Figure  15.  Number 
of  refereed  papers 
per  R&D  dollar 
($M). 
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Figure  16.  Number  of 
refereed  papers  per 
S&E. 
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Figure  17.  Average 
salary  per  S&E 
($K). 
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Figure  17  shows  the  average  salary  paid  per  S&E.  Again,  this  is  an  indica¬ 
tor  of  how  generous  ARL  is  in  promoting  its  S&Es.  Of  those  organizations 
that  would  volunteer  this  particular  data  point  (this  was  an  especially  sen¬ 
sitive  piece  of  information  for  the  private  sector  labs),  ARL  was  clearly  on 
the  low  side,  at  $56K,  and  was  only  slightly  higher  than  the  overall  federal 
average  of  $50.  IK.  Five  of  the  private  sector  labs  (who  requested  that  their 
identities  not  be  revealed)  pay  their  S&Es,  on  average,  over  $75K. 

One  other  type  of  data  that  was  collected  (but  not  plotted  because  of  defi¬ 
nitional  problems)  dealt  with  the  investment  that  a  parent  organization 
invests  in  its  corporate  laboratory.  The  metric  is  R&D  investment  as  a  per¬ 
centage  of  sales.  For  a  government  lab,  "sales"  was  defined  to  be  the  total 
obligational  authority  of  the  agency,  since  this  is  somewhat  comparable  to 
the  amount  of  revenue  taken  in  from  Congress  as  the  "customer."  In  the 
NSF  book's  listing  of  R&D  as  a  percentage  of  sales  for  a  large  number  of 
technology  firms,  we  find  numbers  running  from  10  percent  down  to  2  and 
3  percent.  The  total  industry  average  is  3.1  percent,  the  total  federal  aver¬ 
age  is  6.8  percent,  and  the  DoD  average  is  3.2  percent.  Several  of  the  labs 
specifically  benchmarked  against  in  this  study  were  also  in  the  3  to  6  per¬ 
cent  range.  ARL's  figure  is  only  0.6  percent! 

All  these  data,  including  the  CHI  reports  discussed  earlier,  need  to  be  stud¬ 
ied  in  more  depth.  They  provide  clear  indications,  in  a  general  sense  of 
where  ARL  stands  in  relation  to  its  peers  in  a  variety  of  areas.  Some  of 
these  indications  demonstrate  weaknesses  that  need  attention.  However, 
of  greater  significance  to  our  stakeholders  is  the  fact  that  ARL  is  indeed 
doing  more  with  less,  and  in  some  cases,  a  lot  more  with  a  lot  less. 
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9.  Overhead  Study 

A  second  study  effort  related  to  the  management  of  ARL  was  undertaken 
because  of  the  importance  of  controlling  overhead  costs;  since  these  are  not 
well  understood,  their  management  is  highly  problematic.  Besides  gaining 
knowledge  about  the  subject,  we  were  pleasantly  surprised  to  discover 
that,  compared  to  other  well-known  research  organizations,  both  public 
and  private,  ARL  compares  very  favorably,  keeping  its  overhead  rates  rela¬ 
tively  low. 


9.1  The  Overhead  Problem 

Overhead  is  one  of  the  most  vexing  problems  for  R&D  organizations:  what 
it  is,  and  how  it  is  defined,  accounted  for,  and  most  importantly,  kept 
under  control. 

9.1.1  The  Problem  of  Allocating  Expenses 

Fundamentally,  overhead  is  that  collection  of  expenses  that  cannot  be 
legitimately  charged  to  a  specific  project.  An  engineer  working  at  the 
bench  on  a  project  incurs  direct  costs  for  his  salary,  his  equipment  and  sup¬ 
plies,  and  any  contracts  he  might  award  in  support  of  his  work.  All  charges 
of  the  organization  beyond  that  are,  in  one  sense  or  another,  overhead. 
These  overhead  charges  are  themselves  of  several  types.  The  engineer's  su¬ 
pervisor  (e.g.,  branch  chief)  and  support  staff  (such  as  the  branch  secretary 
and  possibly  some  branch  technicians)  are  billed  as  indirect  overhead. 
These  people  support  the  mission  work  of  the  technical  personnel,  but 
since  they  provide  general  support  to  several  different  projects,  they  are 
not  considered  a  direct  charge  to  any  one  project.  Other  members  of  the  or¬ 
ganization  perform  functions  that  support  the  entire  corporate  enterprise, 
but  have  no  direct  relationship  to  any  specific  project  or  group  of  projects' 
These  functions,  the  General  and  Administrative  (G&A)  overhead,  com¬ 
prise  such  diverse  items  as  corporate  management,  facilities  upkeep,  all 
the  support  functions  (budget,  legal,  procurement,  logistics,  etc),  and  even 
the  writing  of  reports  such  as  this  one. 

An  additional  complication  arises  at  organizations  like  ARL  that  have  mul¬ 
tiple  sites.  Certain  of  the  G&A  costs  are  applicable  to  the  corporation  as  a 
whole,  while  others  are  specific  to  one  site  only.  For  instance,  the  expense 
of  tending  the  grounds  at  the  Adelphi  site  is  charged  to  a  different  account 
from  that  for  tending  the  grounds  at  the  Aberdeen  site.  Thus,  for  the  sake 
of  accuracy  and  fairness  in  billing,  we  have  had  to  divide  our  G&A 
accounts  into  corporate  and  site  G&A.  This  division  leads  to  one  last  twist, 
which  serves  to  increase  the  difficulty  in  accounting  for  our  overhead! 
While  we  are  the  landlord  at  our  Adelphi  site  and,  therefore,  responsible 
for  all  the  expenses  at  that  site,  we  are  tenants  at  all  our  other  sites.  There¬ 
fore,  we  are  billed  for  the  services  we  receive  from  our  hosts.  We  are  billed 
at  different  rates  for  the  same  services  at  different  sites  (including  the  rates 
that  we  charge  ourselves  at  Adelphi),  and  in  some  cases,  certain  services  at 
certain  sites  are  provided  free  of  charge.  This  whole  system  of  overhead  al¬ 
location  is  summarized  in  figure  18. 
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Figure  18.  Types  of 
overhead  costs  in 
relation  to  mission 
(technical)  costs. 
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9.1.2  The  Problem  of  Defining  Terms 

How  are  we  to  calculate  overhead:  in  terms  of  cost  per  work-year,  or  as  a 
cost  percentage  of  funding?  If  the  former,  is  it  the  total  cost  per  direct 
work-year  or  only  the  in-house  costs  per  direct  work-year?  Or  should  the 
denominator  be  total  work-years?  Complicating  these  definitions  further  is 
the  fact  that  a  portion  of  our  G&A  expenses,  roughly  50  percent,  is  paid  for 
by  a  direct  appropriation  into  our  mission  funds.  So  should  this  be  counted 
or  not  when  calculating  total  overhead  "burden"  on  a  direct  labor  hour? 
Depending  on  how  the  word  "overhead"  is  defined,  its  value  can  vary  by  a 
factor  of  three  or  more. 

Benchmarking  overhead  costs  against  similar  organizations  does  not  help 
either,  because  when  it  comes  to  overhead,  no  organizations  are  "similar." 
Definitions  are  different,  host/tenant  relationships  are  different,  business 
and  accounting  practices  are  different,  etc. 

9.2  Overhead  Investigations 
9.2.1  Associate  Director's  Study 

In  July  1994,  our  Associate  Director  conducted  a  detailed  study  of  the  exact 
constituents  of  ARL's  overhead  expense;  obviously,  to  control  and  reduce 
that  expense,  we  first  had  to  understand  what  it  was  and  where  it  came 
from.  This  was  also  a  political  necessity,  since  stakeholders  are  quick  to 
point  the  finger  at  overhead  rates,  even  if  they  do  not  know  precisely  what 
they  are  pointing  at. 

As  part  of  the  1994  study  we  did  some  comparisons  with  NIST,  NRL,  and 
the  Air  Force's  Phillips  Lab — three  organizations  that  could  be  considered 
peers  of  ARL.  Depending  on  which  parameter  was  compared  and  how  it 
was  defined,  ARL  came  out  at  the  bottom,  in  the  middle,  and  at  the  top  in 
terms  of  overhead  costs. 

The  conclusions  of  the  Associate  Director's  study  were  that  overhead  is 
extraordinarily  complex,  there  are  many  different  ways  to  define  and 
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express  it,  ARL  is  more  or  less  comparable  to  similar  organizations  in  its 
rate  of  overhead  expenses,  and  there  are  unquestionably  opportunities  to 
reduce  overhead  everywhere.  Clearly,  trying  to  "level  the  playing  field"  in 
order  to  truly  understand  overhead  expenses  and  compare  them  across 
similar  organizations  is  a  daunting  task. 

9.2.2  GUIRR  Study 

Several  years  ago,  a  scandal"  arose  involving  the  overhead  rates  charged 
by  a  university  to  a  large  corporation  procuring  its  services.  This  led  to  a 
debate  about  who  was  more  "efficient"  (i.e.,  had  the  lowest  overhead  rate) 
in  performing  R&D— academia  or  industry.  In  1992,  as  claims  flew  back 
and  forth,  the  Government-University-Industry  Research  Roundtable 
(GUIRR)  of  the  National  Academies  of  Research  and  Engineering  decided 
to  perform  a  study  to  settle  the  question,  and,  as  a  service  to  the  R&D  com¬ 
munity,  to  shed  some  light  on  this  difficult  problem.  A  group  of  seven 
major  research  universities  and  six  technology  corporations  agreed  to 
devise  a  common  template  for  expressing  their  research  costs  (fig.  19),  and 
then  to  complete  the  data  in  the  template  for  comparison  among  them¬ 
selves.  The  accounting  firm  of  Arthur  Anderson  &  Co.  agreed  to  work  on  a 
pro  bono  basis  to  collect  and  analyze  the  data  so  that  anonymity  could  be 
maintained  to  protect  the  proprietary  aspects  of  the  information. 

After  ARL  completed  the  extensive  in-house  study  described  above  (sect. 
9.2.1),  the  GUIRR  study  came  to  our  attention,  and  in  December  1994,  we 
offered  to  participate  as  a  federal  laboratory  test  bed  for  the  template,  here¬ 
tofore  only  intended  to  be  used  on  academic  and  industrial  labs.  We  were 
accepted  into  the  study  in  March  1995  as  the  first  of  what  ultimately  were 
13  federal  laboratories  in  the  study,  and  made  our  submission  on  the  tem¬ 
plate  for  FY94  data.  We  took  special  care  to  include  all  our  expenses,  in¬ 
cluding  the  self-charged  costs  at  Adelphi,  the  host-charged  costs  where  we 
are  tenants,  and  the  value  of  the  "free"  services  we  received  at  other 
installations  to  the  best  that  we  could  estimate  it. 
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Figure  19.  Form  used  for  GUIRR  overhead  study. 


9.2.3  Results  of  Study 

The  GUIRR  study  was  published  as  an  Arthur  Anderson  report  in  March 
1996 .*  The  final  list  of  33  participants  included  12  additional  federal  labs: 


Academia  (7) 

Indiana  University 

MIT 

UCLA 

University  of 
Pennsylvania 
University  of  Southern 
California 

Vanderbilt  University 
Washington  University 


Industry  (13) 

AT&T 
Coming 
General  Electric 
General  Motors 
Hewlett-Packard 
Phillips  Electronics 
Abbott  Laboratories 
Bristol-Myers  Squibb 
G.D.  Searle 
Genentech 
Monsanto  Corporate 
Research 

Proctor  &  Gamble 
Upjohn 


Federal  government  (13) 

Argonne  Laboratories  (DOE) 

Pacific  Northwest  National  Laboratory  (DOE) 
Brookhaven  National  Laboratory  (DOE) 

John  A.  Volpe  National  Transportation 
Systems  Center  (DOT) 

Lawrence  Berkeley  National  Laboratory 
(DOE) 

Lawrence  Livermore  National  Laboratory 
(DOE) 

Los  Alamos  National  Laboratory  (DOE) 
National  Institute  of  Standards  and  Technol¬ 
ogy  (DOC) 

National  Institutes  of  Health  (HHS) 

Oak  Ridge  National  Laboratory  (DOE) 

Sandia  National  Laboratory  (DOE) 

SRI  International  (a  not-for-profit 
organization) 

U.S.  Army  Research  Laboratory  (DoD) 


The  results  were  surprising.  For  example,  on  average,  the  ratio  of  direct 
charges  to  overhead  (indirect  and  G&A)  was  remarkably  similar  for  all 
three  sectors  roughly  two-thirds  to  one-third,  with  the  federal  sector  sit¬ 
ting  between  industry  (slightly  lower)  and  academia  (slightly  higher).  This 
seems  to  indicate  that,  in  general,  "a  lab  is  a  lab  is  a  lab." 


We  were  pleased  to  see  that  ARL  fared  very  well  in  comparison  with  the 
other  32  participants.  As  figure  20  shows,  in  comparing  the  G&A  portion  of 
the  overhead  costs,  ARL  came  out  below  the  average  for  all  three  sectors. 


Figure  20.  Comparison 
of  overhead  rates  by 
sector. 
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*James  H.  Roth,  " The  Costs  of  Research:  Examining  Patterns  of  Expe?iditures  Across  Research  Sectors," 
Arthur  Anderson  and  Co.  for  the  Government-University-Industry  Research  Roundtable,  Chicago  IL 
11  March  1996.  s 
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9.3  Some  Additional  Thoughts 

"Overhead"  seems  to  have  a  negative  connotation.  However,  overhead  is 
neither  good  nor  bad;  it  is  simply  a  cost  of  doing  business.  This  cost,  like  all 
costs,  should  be  kept  within  reasonable  bounds.  "Reasonable"  is  hard  to 
define,  since  there  are  no  absolute  guidelines,  and  comparisons  with  other 
organizations  are,  as  we  have  seen,  fraught  with  difficulties.  Leveling  the 
playing  field  is  extremely  difficult,  though  the  GUIRR  study  took  a  big 
step  towards  doing  that.  There  are  still  problems,  which  the  Roundtable  is 
considering  pursuing  in  a  follow-on  study.  These  include  the  varying  over¬ 
head  demands  of  different  kinds  of  technologies  (pharmaceuticals  versus 
semiconductors,  for  example)  and  the  fact  that  government  labs  do  not  use 
standard  depreciation  procedures  in  accounting  for  their  expenses. 

Finally,  reducing  overhead  to  zero  is  definitely  not  a  goal  to  be  striven  for. 
In  that  extreme,  there  would  be  no  support  staff,  so  that  S&Es  would  be  left 
to  photocopy  their  own  papers,  procure  their  own  supplies,  mow  their 
own  lawns,  and  mop  their  own  floors,  leaving  them  very  little  time  to  per¬ 
form  their  primary  job:  research.  Rather,  the  goal  should  be  to  keep  over¬ 
head  at  a  "reasonable"  rate,  by  using  such  techniques  as  BPR  (see  sect.  5)  to 
deliver  these  support  services  in  the  most  efficient  and  cost-effective 
manner. 
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10.  Discussion:  Quality  and  Survival 

Following  these  discussions  of  the  various  management  initiatives  that  we 
have  undertaken  and  the  status  of  each,  it  is  appropriate  to  take  a  step  back 
and  consider  certain  aspects  of  the  "big  picture."  One  would  expect  that 
quality  would  be  a  major  contributor  to  the  success  of  any  organization,  re¬ 
search  or  otherwise.  Based  on  this  assumption,  ARL  has  undertaken  a 
number  of  quality  initiatives,  described  briefly  below.  However,  much  to 
our  chagrin,  it  has  been  our  experience  that  quality  alone  is  not  sufficient 
to  assure  survival.  In  fact,  ARL  has  been  deeply  involved  in  a  great  deal  of 
activity  over  the  last  several  months  in  dealing  with  the  survival  problem 
that  faces  most,  if  not  all,  research  organizations,  both  in  and  out  of  gov¬ 
ernment.  Some  of  the  aspects  of  this  problem  have  been  summarized  from 
a  variety  of  documents  and  our  own  experience  over  many  years,  and 
these  are  presented  below. 

10.1  TQM 

10.1.1  The  Quality  "Journey"  at  ARL 

Over  the  past  several  years,  as  TQM  has  waxed  and  waned  as  one  of  a  long 
series  of  management  fads,  ARL  has  been  caught  up  (not  always  willingly) 
in  the  fervor.  As  described  in  section  8,  we  engaged  in  a  fairly  intensive 
benchmarking  effort  with  some  of  the  R&D  world's  corporate  leaders  to 
understand  how  they  used  (or  did  not  use)  TQM  techniques  in  managing 
their  corporate  research  arms.  We  tried  some  experiential  learning,  and 
some  multifunctional,  self-directed  teaming.  These  have  all  had  some  lim¬ 
ited  success,  particularly  in  the  area  of  the  support  functions. 

In  our  efforts  to  implement  TQM,  we  twice  applied  for  the  Presidential 
Quality  Award  (the  federal  government's  counterpart  to  the  Baldrige 
Award),  as  well  as  applying  for  a  State  of  Maryland  quality  award,  and  an¬ 
nually  for  the  Army  R&D  Organization  of  the  Year  Award.  The  exercise  of 
applying  for  awards  was  seen  as  a  way  to  test  our  implementation  of  TQM 
and  reinforce  its  principles  within  the  organization— and  actually  winning 
an  award  would  further  reinforce  the  use  of  TQM.  However,  in  general,  re¬ 
searchers  being  fairly  iconoclastic  people,  the  sloganeering  and  "touchy- 
feelie"  aspects  of  TQM  have  not  met  with  widespread  acceptance.  In  the 
midst  of  our  researchers'  crushing  workload  and  the  continuing  struggles 
to  defend  ARL's  budget,  award  programs  seem  to  add  little  value. 

Although  such  programs  and  processes  may  be  extremely  useful  for  many 
types  of  organizations,  their  value  added  is  more  questionable  for  basic 
research.  Our  Director  s  view  is  that  a  high-quality  research  organization 
does  first-rate,  state-of-the-art  technical  work  and  keeps  its  external  cus¬ 
tomers  happy,  and  that  the  support  staff  does  everything  it  can  to  enable 
the  researchers  to  accomplish  this  goal.  If  we  are  doing  good  work  and 
fully  supporting  our  customers'  needs,  then  the  other  benefits  of  quality 
programs  will  follow  of  their  own  accord. 
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10.1.2  Determining  Quality 

How  can  we  know  if  we  are  doing  good  work  and  satisfying  our  custom¬ 
ers?  Based  on  the  performance  evaluation  techniques  described  in  section 
7,  we  can  use  the  Technical  Assessment  Board  process  to  determine  the 
technical  quality  of  the  work.  The  customer  survey  process  and  the  Stake¬ 
holders'  Advisory  Board  provide  the  feedback  from  our  customers. 

We  have  recently  looked  at  going  beyond  customer  satisfaction  to  "cus¬ 
tomer  value":  that  is,  capturing  the  hearts  of  customers  to  the  point  that 
they  will  not  only  be  satisfied  when  they  receive  your  product,  but  they 
will  go  out  of  their  way,  and  even  sacrifice  time,  money,  or  convenience,  to 
come  back  to  you  the  next  time.  (We  are  all  familiar  with  the  phenomenon 
of  finding  an  auto  mechanic  that  we  trust  to  the  point  that  we  will  even 
pay  more  and  drive  an  extra  distance  just  to  have  that  extra  comfort  factor.) 
ARL  contracted  with  the  University  of  Maryland's  Center  for  Quality  and 
Productivity  to  perform  a  customer  value  survey  among  our  principal  cus¬ 
tomers,  the  RDECs,  to  see  what  attributes  they  would  value  most  from  us. 
The  answer,  like  so  much  that  comes  out  of  the  quality  arena,  was  fairly 
predictable:  communicate  with  the  customers;  talk  to  them,  listen  to  them, 
visit  them,  keep  them  informed,  involve  them  in  the  planning  process,  etc. 

Thus,  we  view  "quality"  less  as  a  process  or  a  mechanism  (or  an  opportu¬ 
nity  to  win  awards)  than  as  a  mindset  that  creates  an  atmosphere  pervad¬ 
ing  all  that  we  do.  It  is  a  prerequisite  if  we  hope  to  survive  and  be  success¬ 
ful  in  fulfilling  our  mission. 

10.2  Survival:  Some  Philosophy  about  "The  R&D  Problem" 

All  corporate  labs  today  (and  ARL  in  particular)  share  a  common  problem: 
a  failure  or  inability  to  communicate  to  stakeholders  (who  usually  do  not 
have  an  R&D  background)  what  R&D  is  all  about,  why  it  is  important,  and 
how  it  is  done.  In  ARL's  experience,  there  are  several  common  manifesta¬ 
tions  of  this  problem,  as  follows. 

10.2.1  "Why  In  House?" 

The  argument  against  an  Army  corporate  lab  is  often  expressed  as  "Let 
industry  (or  the  universities)  do  it.  Why  do  we  have  to  do  it  in  house?" 
This  translates  into  two  questions:  why  do  in-house  research  in  the  Army 
at  all,  and  why  have  an  in-house  central,  or  corporate,  laboratory  for  the 
Army? 

Four  general  arguments  can  be  made  for  in-house  Army  research: 

•  The  "smart  buyer"  argument.  If  the  Army  is  to  be  able  to  intelligently 
acquire  the  complex  technological  developments  for  tomorrow's  battle¬ 
field,  it  must  have  a  cadre  of  people  that  understand  both  the  technology 
and  what  the  private  sector  is  offering  to  deliver,  and  can  then  evaluate 
what  is  delivered  to  assure  that  it  can  do  the  job.  These  knowledgeable 
technology  integrators  must  have  a  place  in  which  to  develop  and  main¬ 
tain  their  expertise  in  militarily  relevant  technologies. 
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•  The  "niche  areas"  argument.  Certain  militarily  relevant  technologies  sim¬ 
ply  have  no  counterpart  in  the  private  sector  (e.g.,  armored  combat 
vehicles).  Therefore,  while  there  are  acquisition  sources  for  things  like 
tanks,  there  is  no  source  of  science  and  technology  for  future  generations 
of  tanks,  other  than  what  is  available  in  house. 

•  The  "high  risk"  argument.  Certain  types  of  research  are  either  of  high  tech¬ 
nical  risk  or  involve/require  the  use  of  very  expensive,  very  large,  and 
very  specialized  capital  equipment.  Industry  is  usually  unwilling  to 
undertake  such  work  or  purchase  such  equipment  on  its  own. 

•  The  need  to  be  close  to  the  user.  Because  personnel  at  an  in-house  Army 
lab  (which  include  some  uniformed  personnel)  will  have  a  closer  relation¬ 
ship  with  the  user  (the  soldier)  than  would  private  sector  personnel,  they 
will  have  a  better  understanding  of  the  user's  needs.  This  allows  the  user's 
requirements  and  concerns  to  be  more  accurately  translated  into  the  acqui¬ 
sition  process. 

Given  in-house  Army  research,  then  why  an  Army  central  laboratory?  The 
reasons  are  the  same  as  can  be  put  forward  for  industry  corporate  labs. 
According  to  a  study  by  the  Industrial  Research  Institute/  90  percent  of  in¬ 
dustrial  technology  corporations  have  a  corporate  or  central  lab,  working 
either  alone  or  in  concert  with  labs  in  the  business  units  or  product  divi¬ 
sions.  These  organizations  have  found  the  following  benefits  of  central 
labs: 

•  Efficiency.  It  would  be  prohibitively  expensive  to  maintain  a  critical  mass 
of  basic  technology  expertise  in  multiple  organizations. 

•  Synergy.  Most  technical  problems  require  multidisciplinary  solutions. 

•  Perspective.  Only  a  central  lab  can  pursue  paradigm-breaking  approaches 
to  achieve  revolutionary  improvements  in  effectiveness  (in  ARL's  case, 
battlefield  effectiveness). 

10.2.2  "Culture  Clash"  Between  Leadership  and  R&D 

The  culture  of  the  military  leadership  who,  by  and  large,  have  had  only 
minimal  (if  any)  scientific  training  is  significantly  different  from  that  of  the 
R&D  environment.  The  leadership  tends  to  have  an  understandable  dis¬ 
trust  of  a  system  that  continually  says,  "Send  money,  have  faith.  Some¬ 
thing  good  will  result  in  20  or  30  years,  although  we're  not  exactly  sure 
what  it  will  be.  Although  this  is  indeed  the  way  R&D  usually  works 
(strange  as  it  sounds),  it  is  obviously  a  difficult  argument  to  make  to 
decision-makers  faced  with  severe  cutbacks  in  the  total  budget.  The  prob¬ 
lem  is  how  to  communicate  to  the  leadership  the  true  worth  of  R&D  and 
the  benefits  of  investing  in  it.  This  problem  is  compounded  by  the  fact  that 
the  fruits  of  6.1  and  6.2  work  tend  not  to  appear  in  any  obvious  way  in  the 
final  product.  Even  though  the  technologies  evolved  over  the  past  20  or  so 
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years  have  made  possible  the  current  generation  of  (for  example)  tanks, 
the  ARL  logo  does  not  appear  anywhere  on  the  M1A2  Abrams  tank.  To 
deal  with  this  problem,  ARL  prepared  and  distributed  a  marketing  bro¬ 
chure  called  "ARL  Within"  (emulating  the  "Intel  inside"  advertising  cam¬ 
paign).  It  seeks  to  reveal  the  many  places  where  ARL  technologies  appear 
in  today's  fielded  systems.  However,  even  this  approach  risks  inspiring  the 
question,  "What  have  you  done  for  me  lately?"  and  brings  us  back  to  the 
problem  of  the  time  lag  between  inputs  and  outcomes.  As  described  in  sec¬ 
tion  7,  there  are  no  metrics  to  allow  a  real-time  evaluation  of  a  laboratory's 
performance  in  terms  of  outcomes.  We  faced  this  as  part  of  our  GPRA  pilot 
project  by  the  construction  of  our  Performance  Evaluation  Construct  (sect. 
7).  Although  this  has  drawn  a  great  deal  of  interest  from  all  over  the  gov¬ 
ernment  (as  other  labs  struggle  with  the  requirements  of  GPRA),  it  still 
does  not  do  much  for  our  military  stakeholders. 

Even  if  the  senior  leadership  is  not  hostile  to  an  in-house  corporate  labora¬ 
tory,  if  often  sends  ambiguous  or  ambivalent  signals.  For  example,  during 
the  first  meeting  of  our  Stakeholders'  Advisory  Board,  we  discussed, 
among  other  things,  the  various  "balances"  that  we  are  trying  to  achieve  as 
a  research  laboratory  (between  in  and  out  of  house  work,  between  mission 
and  customer-funded  work,  etc).  One  of  the  balances  discussed  was 
among  near-,  mid-,  and  far-term  work.  We  estimated  the  balance  at  45,  34, 
and  21  percent,  respectively.  After  a  short  discussion,  the  consensus  was 
that  we  were  out  of  balance,  and  that  our  far-term  work  should  be  50  per¬ 
cent.  Although  this  is  obviously  closer  to  the  "right"  answer,  it  flies 
squarely  in  the  face  of  the  other  signals  we  continually  receive  from  the 
Pentagon. 

10.2.3  The  " Bananas "  Problem 

A  currently  fashionable  suggestion  is  that  the  Army  has  "all  the  technology 
it  needs"  for  now.  Thus,  we  could  call  a  moratorium  on  research  for  five 
years  and  use  up  what  we  have  now,  especially  since  the  Army  is  not  cur¬ 
rently  procuring  any  new  systems  that  could  use  the  technology  we  are 
developing.  At  the  end  of  five  years,  we  could  check  back,  see  how  we're 
doing,  and  then,  if  we  need  some  more  technology,  start  some  of  these  labs 
up  again.  Within  the  military,  some  people  are  calling  this  a  "tactical 
pause." 

Aside  from  the  obvious  problems  of  shutting  down  and  later  reopening 
research  laboratories,  what  is  often  not  recognized  is  that  research  is  much 
like  bananas.  If  I  deliver  to  you  now  all  the  bananas  that  you  need  for  the 
next  five  years,  you  presumably  won't  need  to  invest  in  any  more  bananas 
for  that  period  of  time.  Then,  I'll  come  by  in  a  few  weeks  and  check  on  how 
you  and  your  five  years'  worth  of  bananas  are  doing. 

The  point  is,  bananas  don't  keep;  neither  does  research.  Although  research 
results  do  not  actually  rot,  they  become  out  of  date,  superseded  by  other 
research,  or  rendered  irrelevent  by  new  directions  in  research.  (Further¬ 
more,  closing  down  labs  is  more  like  uprooting  your  banana  trees  than 
merely  trying  to  freeze  the  bananas.) 


71 


10.2.4  The  Portfolio  Problem 

We  are  often  asked  how  we  know  what  to  work  on.  The  answer  is,  of 
course,  complex.  We  basically  try  to  sense  the  environment.  We  talk  to  our 
customers,  both  the  immediate  customers— the  developers  (to  whom  we 
deliver  products) — and  the  ultimate  customers  (the  soldiers),  who  end  up 
using  the  products  that  our  immediate  customers  develop  and  field.  We 
maintain  contact  with  the  larger  scientific  community  to  understand  the 
latest  technologies  being  worked  on  elsewhere;  we  also  perform  work  that 
evolves  naturally  as  a  result  of  the  path  that  research  lays  out  in  front  of  us. 
We  have  a  strategic  planning  process  that  sets  the  major  technical  vectors 
for  the  laboratory.  Another  approach  we  have  taken  was  the  promulgation 
of  the  five  Grand  Challenges  (discussed  in  sect.  7.2.1),  as  a  means  to  dem¬ 
onstrate  (and  actually  provide)  focus  to  the  lab's  efforts. 

This  approach  also  seemed  to  resonate  with  the  military  leadership.  How¬ 
ever,  despite  our  efforts,  there  is  always  some  uneasiness  about  why  we 
have  chosen  to  do  one  piece  of  work  and  not  another. 

10.2.5  The  "Duplication"  Illusion 

Everybody  is  doing  the  same  thing!"  Among  senior  leaders  whose  train- 
ing  and  career  experience  is  not  technical,  misunderstandings  can  arise. 
After  a  quick  scan  of  the  landscape,  they  observe  that  everyone  is  working 
on  the  same  problem.  They  imagine  enormous  savings  from  streamlining 
all  this  "duplication."  An  extreme  case  of  this  kind  of  thinking  occurred 
several  years  ago,  when  one  of  these  senior  leaders  was  visiting  several  of 
the  Army's  laboratories  and  observed  that  there  were  lasers  to  be  found 
everywhere.  He  concluded  that  we  were  all  duplicating  each  other's  laser 
research.  Certain  organizations  were  subsequently  "forbidden"  to  work 
with  lasers.  We  tried  to  explain  that  all  these  lasers  were  laboratory  instru¬ 
ments  and  were  being  used  as  tools  in  other  types  of  research,  much  as  one 
would  use  a  hammer  or  a  screwdriver.  Our  argument  had  no  impact;  his 
mind  was  made  up,  and  he  did  not  wish  to  be  confused  by  the  facts. 

10.2.6  Defending  Research 

The  five  situations  described  above  are  some,  but  by  no  means  all,  of  the 
manifestations  of  the  broader  problem  of  how,  in  an  era  of  downsizing,  a 
corporate  laboratory  maintains  the  support  of  its  stakeholders.  These  prob¬ 
lems  are  not  particularly  new  or  unusual.  Laboratories  in  both  the  public 
and  private  sectors  have  struggled  with  them  for  decades  without  having 
found  any  universal  solution.  ARL  has  also  been  engaged  in  that  struggle. 
We  do  not  claim  to  have  found  that  solution  either,  other  than  to  use  every 
bit  of  creativity  we  can  muster  to  communicate  with  our  stakeholders,  ex¬ 
plain  who  we  are  and  why  we  are  important,  market  them,  persuade  them, 
beg,  plead,  and  cajole.  However,  it  is  an  unending  labor. 
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11.  Conclusion 


How  are  we  doing?  Have  we  made  any  real  progress  in  managing  a  corpo¬ 
rate  research  laboratory?  We  think  so,  but  we  have  a  long  way  to  go.  Here 
is  the  "warts  and  all"  report  card  as  promised  at  the  beginning  of  this 
report: 

•  FedLab.  This,  the  real  reinvention  of  our  fundamental  mission,  has  been 
under  way  for  two  and  half  years,  and  is  looking  good.  Very  exciting  tech¬ 
nical  work  is  going  on  at  this  point.  However,  the  question  is  whether  there 
will  be  some  real,  transitionable  output  at  the  end  of  the  five  years.  Just  as 
with  all  research  programs,  we  must  wait  and  see.  Some  rough  administra¬ 
tive  edges  still  need  smoothing  out,  but  these  are  not  show  stoppers.  One 
problem,  which  is  now  getting  serious  attention,  is  better  integration  of  the 
work  among  the  partners  in  each  consortium.  Grade:  A  (plus  a  Hammer 
Award). 

•  Open  Laboratory.  Managing  the  support  structure  to  facilitate  the  FedLab 
initiative,  especially  the  personnel  rotation  aspect,  the  Open  Laboratory 
has  surfaced  lots  of  problems,  but  not  tackled  many.  While  Open  Lab  is  not 
absolutely  critical  for  the  success  of  FedLab,  without  it  the  working  envi¬ 
ronment  will  suffer.  These  are  complex  administrative  and  cultural 
problems  for  the  most  part,  and  will  take  enormous  willpower  to  fix. 
Grade:  C-. 

•  Waivers.  One  part  of  reinvention  is  unburdening  ourselves  from  inappro¬ 
priate,  irrelevant,  outdated,  or  just  plain  silly  regulations  and  policies  from 
higher  headquarters.  Although  it  is  often  discussed  as  if  it  were  the  same 
as  reinvention,  getting  these  waivers  is  mostly  nibbling  around  the  mar¬ 
gins  of  the  serious  problems  facing  us.  However,  we  have  been  able  to 
make  some  progress  in  gaining  small  reliefs  here  and  there,  which  result  in 
time  and  process  savings  in  many  of  the  support  areas.  Several  of  the 
waivers,  current  and  planned,  are  important  to  enable  certain  aspects  of 
both  FedLab  and  Open  Lab.  Although  not  at  all  insignificant,  it  is  not  the 
"be  all  and  end  all"  that  it  was  advertised  as.  Grade:  B. 

•  Business  Process  Reengineering  (BPR).  Similar  to  the  waivers  initiative, 
BPR  is  also  aimed  at  unburdening  ourselves,  but  this  time  from  self- 
imposed  constraints.  A  lot  of  good  work  was  accomplished  in  analyzing 
what  could  be  done,  and  a  lot  of  potential  was  defined.  However,  much  of 
the  implementation  depends  on  automated  information  systems  that  we 
do  not  yet  have  (but  which  are  starting  to  appear).  There  is  also  some  cul¬ 
tural  resistance  already  forming  about  changing  the  way  we  deliver  some 
of  our  support  services.  Grade:  A  for  preparation,  and  C  for  implementa¬ 
tion. 

•  Laboratory  Quality  Improvement  Program  (LQIP).  The  follow-on  to  Lab 
Demo,  LQIP  was  to  provide  the  large-scale  reliefs  that  every  one  of  the 
dozens  of  blue  ribbon  panels  over  the  past  30  years  have  said  are  necessary 
for  the  effective  functioning  of  the  DoD  laboratories.  Despite  a  few  small 
successes,  LQIP  was  basically  stymied  at  every  turn  whenever  one  of  the 
really  "hard"  problems  was  raised  (such  as  personnel  management  or  fis- 
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cal  resource  management).  Grade:  At  the  last  major  LQIP  meeting,  the  dis¬ 
cussion  was  whether  the  grade  should  be  a  D-  or  an  F. 

•  Alternative  Personnel  System  Demonstration.  The  single  most  important 
element  of  all  laboratory  management  reinventions,  reforming  the  person¬ 
nel  system  has  been  called  for  by  panels  of  experts  for  decades.  Although 
the  most  significant  features  of  the  proposal  were  removed  by  DoD,  the 
program  is  now  up  and  running.  Grade:  A  (for  potential)  (plus  a  Hammer 
Award). 

•  GPRA.  Under  GPRA,  we  devised  new  techniques  for  planning  and  evalu¬ 
ating  R&D. 

—  Planning.  ARL  established  a  business  planning  process  that  goes  from 
strategic  plan  to  annual  performance  planning  and  reporting,  coupled 
to  the  budget  cycle.  The  four-volume  business  plan  itself  is  well  put 
together  and  breaks  some  new  ground  by  including  specific  strategic 
technical  goals  (instead  of  just  the  usual  "motherhood"  statements). 
However,  the  linkage  between  the  volumes  is  not  as  clear  as  it  should 
be,  and  deployment  of  the  overall  plan  down  throughout  the  workforce 
has  not  been  accomplished  to  the  extent  that  it  should  be.  Grade:  B. 

-  Evaluation.  We  devised  and  implemented  the  three-pillared  Perfor¬ 
mance  Evaluation  Construct,  consisting  of  peer  review,  customer  feed¬ 
back,  and  metrics.  The  metrics  pillar  has  suffered  from  the  lack  of  a 
corporate  information  system.  However,  in  our  opinion  this  is  still  the 
best  that  anyone  has  come  up  with  to  date,  in  or  out  of  government. 
Grade:  A+. 

•  Benchmarking.  Benchmarking  provides  a  calibration  of  where  ARL  stands 
compared  to  its  peers.  Although  not  rigorous,  nor  in  depth,  the  various 
benchmarking  efforts  have  given  clear  indications  of  how  well  we  are  do- 
ing  in  certain  areas  of  interest  and  of  where  improvements  are  necessary. 
Grade:  B+. 

•  Overhead  Study.  We  analyzed  exactly  what  overhead  is  in  terms  of  a 
research  organization,  and  identified  where  and  how  it  can  be  reduced. 
The  study  gave  us  further  insight  into  managing  the  lab.  Grade:  A+. 

This  is  a  progress  report,  not  a  final  report.  All  the  initiatives  that  have 
been  discussed  are  still  under  way,  and  much  work  remains  to  be  done.  All 
in  all,  we  believe  that  ARL  has  come  a  long  way  in  tackling  the  problem  of 
managing  research.  We  compare  favorably  in  many  areas  with  our  world- 
class  peers,  and  in  some  areas  we  believe  we  are  the  best  in  class.  In  other 
areas,  we  have  much  work  to  do.  We  have  made  a  great  effort  to  share 
what  we  have  learned  throughout  the  defense  and  the  federal  laboratory 
community.  Managing  R&D  organizations  is  an  extremely  complex  under¬ 
taking.  Thus,  we  sometimes  say  that  in  many  of  these  areas,  we  are  twice 
as  good  as  anyone  else:  on  a  scale  of  1  to  10,  we  are  a  2! 

We  do  not  believe  that  we  have  all  the  answers.  However,  we  do  think  that 
we  have  figured  out  what  most  of  the  questions  are. 
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12.  Epilogue:  Outsourcing 

As  this  report  goes  to  press,  ARL  is  about  to  embark  on  another  major  ini¬ 
tiative  that,  along  with  FedLab,  will  complete  the  transformation  into  the 
federal  R&D  organization  of  the  future.  Driven  by  increasing  fiscal  pres¬ 
sures  and  by  Defense  Reform  Initiative  Directive  20  (DRID-20),  we  are  en¬ 
gaged  in  an  intensive  study  of  the  options  available  to  us  for  outsourcing 
major  functions  of  the  lab.  We  have  formed  a  project  management  office 
and  engaged  a  contractor  and  a  panel  of  expert  advisors  to  assist  us  in 
studying  the  feasibility  of  the  various  outsourcing  approaches.  These  in¬ 
clude,  among  others, 

•  A-76  commercial  activities  competition, 

•  Employee  Stock  Ownership  Programs  (ESOPs), 

•  a  Government-Owned  Contractor-Operated  (GOCO)  organization,  and 

•  a  variety  of  other  partial  or  total  privatization  schemes. 

The  idea  is  to  take  those  functions  that  are  considered  to  be  commercial  ac¬ 
tivities  (as  opposed  to  those  core  business  activities  that  constitute  inher¬ 
ently  governmental  functions),  and  outsource  them,  either  function  by 
function,  or  as  a  single  entity  that  will  then  supply  these  functions  to  ARL 
on  some  sort  of  contractual  basis.  We  have  been  benchmarking  as  we  pro¬ 
ceed  with  other  organizations  that  have  done  this,  both  in  the  government 
(such  as  NIH's  National  Cancer  Laboratory  and  OPM's  security  investiga¬ 
tions  unit),  and  outside  (the  UK's  DERA). 

This  process  raises  exceptionally  complex  issues,  not  the  least  of  which  is 
how  one  defines  "commercial  activity"  versus  "inherently  governmental 
function."  The  study  is  ongoing,  so  there  are  no  results,  or  even  guesses,  as 
to  how  it  is  going  to  turn  out.  The  only  thing  that  is  certain  is  that  some 
course  of  action  will  be  taken  over  the  next  six  months,  and  that  ARL's 
metamorphosis  into  a  truly  world-class  research  organization  with  a  state- 
of-the-art  management  program  will  be  completed  over  the  next  year  or 
so. 


“Plan  for  change — it  happens!  ” 
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